-- Logs begin at Thu 2022-05-19 11:18:23 UTC, end at Thu 2022-05-19 12:51:43 UTC. -- May 19 11:56:11.555755 dv-cinder-804035-46-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 19 11:56:11.564410 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 11:56:11 2022] *** May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: nodename: dv-cinder-804035-46-iscsi May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: machine: x86_64 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: clock source: unix May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: pcre jit disabled May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: detected number of CPU cores: 4 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: current working directory: / May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: detected binary path: /usr/bin/uwsgi-core May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: your processes number limit is 47680 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: your memory page size is 4096 bytes May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: detected max file descriptor number: 2048 May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: lock engine: pthread robust mutexes May 19 11:56:11.566676 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: thunder lock: enabled May 19 11:56:11.567672 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 19 11:56:11.567672 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 11:56:11.610540 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: Python main interpreter initialized at 0x55b23a02c9c0 May 19 11:56:11.610540 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: python threads support enabled May 19 11:56:11.610540 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: your server socket listen backlog is limited to 100 connections May 19 11:56:11.610540 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: your mercy for graceful operations on workers is 90 seconds May 19 11:56:11.610892 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: mapped 403077 bytes (393 KB) for 2 cores May 19 11:56:11.610892 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: *** Operational MODE: preforking *** May 19 11:56:11.611190 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: *** uWSGI is running in multiple interpreter mode *** May 19 11:56:11.611190 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: spawned uWSGI master process (pid: 103107) May 19 11:56:11.611371 dv-cinder-804035-46-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 19 11:56:11.611910 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: spawned uWSGI worker 1 (pid: 103108, cores: 1) May 19 11:56:11.614380 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: spawned uWSGI worker 2 (pid: 103109, cores: 1) May 19 11:56:11.614380 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 11:56:12.776452 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 19 11:56:12.777076 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 19 11:56:13.489733 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:13.489733 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: from cryptography.utils import int_from_bytes May 19 11:56:13.490119 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:13.490119 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: from cryptography.utils import int_from_bytes May 19 11:56:13.501100 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:13.501100 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: from cryptography.utils import int_from_bytes May 19 11:56:13.501433 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:13.501433 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: from cryptography.utils import int_from_bytes May 19 11:56:16.039997 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 11:56:16.040580 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 11:56:16.040756 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 11:56:16.041107 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 11:56:16.041359 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 11:56:16.041740 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.042053 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.042365 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.042699 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.042998 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.043292 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.043603 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.043913 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.044231 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.044468 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.044796 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.045114 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-804035-46-iscsi {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.045424 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.045724 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.046024 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.046334 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.046612 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.046929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.047024 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 11:56:16.047385 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 11:56:16.047471 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.047697 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.047807 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 11:56:16.048237 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 11:56:16.048473 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.048587 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 11:56:16.048802 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.048934 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049109 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049222 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049441 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049534 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049747 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.049851 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050048 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050135 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050365 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050495 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050666 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050753 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.050975 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051068 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051274 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051373 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051539 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051661 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-804035-46-iscsi {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051848 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.051981 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.052165 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-804035-46-iscsi {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.052286 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.052508 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.052592 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.052832 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053022 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053123 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053327 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053435 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053628 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053733 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.053939 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054032 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054253 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054348 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054552 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054810 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.054921 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055117 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055202 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055414 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055568 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055716 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.055802 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056036 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056133 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056335 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056511 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056633 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056748 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.056966 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057051 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057262 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057350 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057557 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057643 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057801 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.057887 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058100 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058231 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-804035-46-iscsi {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058410 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058540 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058721 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.058848 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059034 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059156 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059281 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059529 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059621 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059830 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.059924 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060134 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060219 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060430 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060515 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060755 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.060844 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061056 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061156 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061354 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061440 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061650 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061735 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.061947 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062089 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062239 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062335 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062532 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062623 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062831 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.062926 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063132 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063220 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063441 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063535 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063736 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.063837 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064028 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064124 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064273 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064418 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064573 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064688 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.064918 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065004 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065232 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065325 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065548 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065648 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065795 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.065888 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066042 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066137 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066352 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066440 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066662 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.066964 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067062 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067263 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067357 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067563 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067659 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.067936 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068036 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068211 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068297 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068463 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068549 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068783 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.068889 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069080 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069174 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069390 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069483 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069687 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069772 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.069984 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070069 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070281 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070366 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070572 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070657 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070868 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.070953 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071162 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071256 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071460 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071555 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071755 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.071852 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072052 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072137 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072346 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072442 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072687 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072790 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.072961 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073057 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073259 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073344 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073553 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073638 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073855 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.073955 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074150 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074249 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074446 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074532 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074742 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074841 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.074990 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075074 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075282 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075367 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075578 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075672 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075887 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.075972 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076185 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076270 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076484 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076580 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076803 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.076901 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077093 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077181 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077390 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077485 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077685 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077770 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.077982 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078068 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078270 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078363 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078563 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078647 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078856 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.078941 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079150 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079243 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079443 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079537 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079732 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.079815 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080026 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080111 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080320 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080405 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080616 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080740 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.080934 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081021 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081225 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081309 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081515 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081601 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081824 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.081923 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082119 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082204 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082418 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082555 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082698 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082832 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.082996 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.083129 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.083288 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.083415 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.083623 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.083753 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.083921 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.084050 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.084219 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.084361 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.084529 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.084697 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.084836 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.084966 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.085136 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.085288 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.085442 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.085571 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.085739 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.085878 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.086036 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.086174 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.086337 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.086489 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.086630 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.086721 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.086925 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.087014 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.087221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.087307 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.087516 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.087601 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.087812 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.087899 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.088105 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.088190 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.088414 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.088508 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.088747 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.088838 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.089052 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.089137 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.089351 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.089435 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.089649 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.089741 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.089958 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:16.090044 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090265 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090364 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090565 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090650 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090897 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.090983 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091208 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091320 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091523 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091609 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091837 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.091930 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092145 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092248 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092446 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092542 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092767 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.092869 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093071 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093167 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093382 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093532 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093685 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093815 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.093987 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094134 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094295 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094428 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094593 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094719 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.094896 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095034 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095197 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095326 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095499 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095628 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095799 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.095933 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096100 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096232 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096409 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096541 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096743 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.096872 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097005 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097137 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097317 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097456 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097622 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097757 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.097924 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098058 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098224 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098357 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098518 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098648 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098811 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.098942 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099177 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099326 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099496 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099638 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099794 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.099925 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100105 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100246 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100425 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100559 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100752 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.100880 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101065 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101209 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101357 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101488 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101648 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101784 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.101950 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102088 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102248 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102380 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102545 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102686 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102846 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.102988 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103147 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103290 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103442 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103601 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103808 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.103953 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104086 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104217 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104349 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104480 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104620 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104792 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.104930 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105062 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105211 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105353 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105504 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105649 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105789 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.105922 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106055 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106194 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106339 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106491 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106620 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106751 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.106974 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107187 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107342 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107612 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107742 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.107882 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108023 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108161 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108291 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108430 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108562 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108723 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.108879 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109013 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109152 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109283 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109415 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109583 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109738 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.109869 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110006 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110181 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110331 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110470 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110600 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110749 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.110887 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111026 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111157 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111296 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111426 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111627 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111827 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.111972 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112114 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112258 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112389 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112520 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112607 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112785 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.112929 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113073 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113214 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113346 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113478 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113610 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113758 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.113902 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114035 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114205 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114355 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114507 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114649 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.114882 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115016 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115197 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115335 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115507 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115604 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115818 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.115911 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116112 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116203 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116417 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116511 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116735 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.116824 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117032 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117128 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117329 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117422 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117629 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117714 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.117925 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118010 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118222 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118308 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118517 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118609 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118814 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.118907 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119112 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119198 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119411 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119508 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119715 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.119799 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120006 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120090 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120296 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120380 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120592 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120741 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.120948 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121034 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121249 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121334 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121561 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121662 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121861 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.121947 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122168 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122254 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122469 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122563 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122769 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.122854 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123067 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123159 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123367 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123464 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123672 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123802 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.123968 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124060 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124266 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124360 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124564 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124666 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124897 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.124983 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125195 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125280 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125497 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125588 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125789 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.125873 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126099 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126196 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126406 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126490 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126701 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.126793 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127008 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127094 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127309 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127402 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127623 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127709 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.127926 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128020 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128226 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128319 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128525 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128617 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128846 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.128933 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129144 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129445 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129541 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129744 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.129836 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130042 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130128 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130363 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130466 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130684 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130769 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.130991 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131076 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131292 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131459 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131585 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131876 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.131960 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132176 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132263 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132468 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132559 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132796 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.132896 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133108 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133194 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133420 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133507 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133716 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.133801 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134010 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134096 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134319 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134411 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134614 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134705 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134906 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.134993 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135199 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135294 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135493 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135578 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135790 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.135875 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136094 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136179 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136397 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136495 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136718 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.136874 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137041 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137128 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137340 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137571 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137664 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.137928 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138014 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138227 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138320 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138536 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138624 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138839 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.138924 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.139137 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.139438 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.139537 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.139737 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.139823 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140036 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140121 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140334 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140421 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140680 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.140967 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141068 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141268 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141354 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141569 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141668 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141863 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.141948 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.142159 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.142250 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.142456 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.142542 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.142756 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143008 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143115 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143333 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143421 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143640 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143729 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.143948 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144044 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144253 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144348 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144550 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144664 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144885 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.144973 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145189 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145275 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145490 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145588 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145799 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.145909 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146126 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146215 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146419 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146504 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146713 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.146828 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.147013 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.147106 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.147302 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.147395 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.147913 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148015 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148206 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148300 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148505 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148590 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148827 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.148916 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149121 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149206 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149413 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149498 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149701 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.149786 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150002 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150096 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150294 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150554 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150648 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150871 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.150956 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151178 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151261 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151473 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151567 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151765 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.151852 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152055 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152141 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152347 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152433 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152695 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152785 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.152996 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153081 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153318 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153419 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153620 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153707 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.153920 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154019 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154331 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154507 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154602 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154807 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.154892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155101 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155265 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155404 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155489 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155690 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155808 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.155981 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156087 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156276 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156363 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156565 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156671 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156882 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.156980 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157180 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157272 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157471 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157555 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157766 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.157851 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158057 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158151 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158354 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158441 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158644 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158738 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.158936 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159028 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159232 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159317 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159520 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159612 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159811 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.159906 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160111 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160199 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160407 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160495 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160721 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.160811 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161024 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161127 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161315 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161401 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161671 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161773 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.161967 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162062 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162261 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162347 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162554 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162659 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162844 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.162938 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163142 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163226 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163433 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163518 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163728 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.163826 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164030 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164116 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164321 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164415 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164615 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164734 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.164935 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165020 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165224 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165309 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165515 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165600 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165805 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.165889 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166111 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166250 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166392 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166526 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166686 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166816 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.166978 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167111 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167264 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167405 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167547 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167686 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167820 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.167960 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168091 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168231 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168364 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168496 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168669 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168825 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.168956 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169100 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169239 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169371 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169523 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169661 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169833 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.169981 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170113 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170243 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170406 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170548 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170689 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170818 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.170962 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171094 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171280 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171419 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171558 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171688 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171859 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.171995 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172144 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172276 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172430 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172571 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172752 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.172893 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173063 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173199 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173343 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173472 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173643 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173785 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.173937 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174071 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174203 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174334 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174501 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174599 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174798 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.174882 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175092 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175187 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175380 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175474 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175672 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175758 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.175962 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176048 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176255 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176357 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176548 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176655 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176883 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.176966 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177340 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177498 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177585 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177792 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.177886 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178086 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178174 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178383 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178466 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178681 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178767 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.178981 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179068 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179282 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179368 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179582 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179667 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179881 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.179967 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180179 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180265 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180476 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180560 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180796 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.180884 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181088 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181173 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181385 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181483 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181678 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181764 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.181976 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182060 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182272 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182433 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182562 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182656 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182861 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.182954 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183153 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183247 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183535 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183741 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.183825 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184058 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184152 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184370 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184456 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184681 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184770 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.184990 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185077 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185290 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185375 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185591 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185678 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185892 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.185978 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186193 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186277 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186486 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186570 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186787 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.186872 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187090 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187176 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187391 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187484 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187694 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187782 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.187991 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188084 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188291 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188376 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188590 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188690 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.188912 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189006 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189209 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189315 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189507 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189593 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189799 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.189891 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190098 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190197 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190391 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190482 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190688 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190772 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.190981 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191075 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191373 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191457 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191667 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191760 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.191969 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192053 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192262 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192352 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192561 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192696 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.192939 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193027 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193252 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193337 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193561 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193647 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.193951 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194171 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194255 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194475 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194567 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194790 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.194877 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195107 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195193 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195408 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195493 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195705 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.195800 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196005 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196089 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196317 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196416 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196610 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196746 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.196929 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197014 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197225 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197323 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197541 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197701 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.197991 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198131 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198265 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198497 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198638 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198810 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.198964 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199090 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199385 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199528 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199689 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199832 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.199985 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200119 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200282 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200417 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200595 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200762 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.200916 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201240 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201377 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201510 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201643 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201787 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.201920 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202078 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202207 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202340 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202474 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202644 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202779 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.202924 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203065 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203235 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203378 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203540 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203674 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203852 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.203947 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204163 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204250 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204459 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204555 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204795 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.204907 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205097 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205203 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205389 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205490 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205684 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205806 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.205983 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206124 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206314 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206409 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206610 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206696 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.206921 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207007 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207238 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207332 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207554 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207639 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207852 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.207936 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.208155 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.208277 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.208484 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.208570 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.208960 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209049 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209270 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209400 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209575 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209706 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.209910 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210039 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210211 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210298 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210509 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210824 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.210974 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211128 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211212 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211422 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211505 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211721 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.211806 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212015 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212098 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212307 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212390 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212600 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212735 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.212922 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.213005 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.213221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.213307 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.213519 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.213604 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215055 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215169 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215315 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215407 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215407 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215612 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215726 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215810 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215885 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.215965 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216045 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216124 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216190 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216314 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216405 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216472 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216601 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216723 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216848 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.216848 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.217124 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.217431 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.217734 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.217878 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.218040 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220343 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220442 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220537 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220620 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220755 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220836 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220913 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.220990 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221083 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221163 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221250 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221329 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221407 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221484 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221570 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221649 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221734 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221734 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221877 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.221954 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222081 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222162 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222240 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222240 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222388 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222476 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222563 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222642 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222642 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.222922 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.223223 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.223524 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.223826 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224186 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224270 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224504 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224595 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224847 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.224958 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225172 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225258 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225480 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225580 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225785 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.225872 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226095 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226227 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226398 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226553 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226745 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.226876 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227050 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227205 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227345 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227517 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227648 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227842 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.227966 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228162 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228259 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228475 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228608 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228813 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.228940 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229127 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229288 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229438 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229525 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229748 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.229836 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230058 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230250 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230348 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230438 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230659 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230754 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.230969 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231065 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231279 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231364 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231589 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231804 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.231900 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.232128 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.232216 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.232439 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.232525 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.232848 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233024 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233162 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233339 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233472 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233668 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233780 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.233989 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234077 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234309 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234395 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234622 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234787 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.234923 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235053 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235246 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235385 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235544 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235649 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235845 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.235935 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236149 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236238 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236452 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236618 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236782 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.236913 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237094 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237229 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237395 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237540 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237690 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.237902 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238052 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238211 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238359 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238503 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238791 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.238925 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239099 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239240 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239405 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239539 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239730 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.239864 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240061 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240201 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240376 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240519 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240700 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.240842 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241018 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241168 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241316 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241402 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241609 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241699 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.241912 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242007 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242214 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242300 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242509 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242595 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242820 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.242926 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243108 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243406 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243497 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243701 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.243790 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244001 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244205 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244291 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244515 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244679 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244877 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.244964 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245189 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245287 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245484 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245579 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245786 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.245873 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246087 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246181 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246382 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246469 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246677 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246762 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.246970 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247066 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247271 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247357 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247562 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247648 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.247943 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248151 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248239 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248450 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248538 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248765 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.248855 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249063 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249149 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249367 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249452 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249705 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249888 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.249999 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250229 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250316 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250539 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250626 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.250946 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251169 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251254 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251471 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251558 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251768 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.251853 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252081 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252167 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252380 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252466 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252695 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.252922 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253019 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253247 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253333 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253561 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253648 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253856 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.253940 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254159 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254246 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254462 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254549 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254759 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.254849 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255056 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255143 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255393 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255516 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255691 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255776 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.255996 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256082 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256292 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256377 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256588 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256707 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.256914 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257218 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257328 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257509 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257605 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257805 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.257903 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258106 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258219 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258402 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258489 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258694 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258790 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.258991 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259133 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259290 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259431 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259618 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259753 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.259912 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260053 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260215 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260347 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260518 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260685 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260849 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.260937 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261150 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261236 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261443 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261529 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261738 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.261823 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262037 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262130 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262339 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262433 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262640 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262725 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.262941 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263041 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263231 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263318 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263568 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263652 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.263923 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264088 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264231 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264413 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264542 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264748 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.264938 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265110 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265252 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265388 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265564 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265688 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265871 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.265985 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266182 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266289 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266483 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266604 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266802 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.266889 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267134 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267221 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267453 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267539 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267764 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.267849 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268058 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268151 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268354 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268452 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268655 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268756 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.268969 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269054 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269268 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269363 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269564 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269657 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269858 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.269945 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270152 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270247 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270452 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270539 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270745 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.270832 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271150 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271244 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271608 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271828 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.271989 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.272134 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.272416 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.272506 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273068 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273168 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273376 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273473 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273672 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273766 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.273973 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274058 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274282 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274377 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274603 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274689 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274900 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.274986 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275197 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275293 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275490 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275606 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275786 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.275910 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276084 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276222 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276406 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276536 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276755 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.276865 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277067 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277180 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277380 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277493 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277682 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277797 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.277993 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278116 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278296 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278419 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278607 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278741 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.278920 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279072 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279226 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279390 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279525 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279706 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.279849 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280003 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280152 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280300 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280457 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280596 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280784 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.280924 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281087 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281236 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281389 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281537 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281692 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.281830 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282011 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282125 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282328 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282415 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282625 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282711 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.282939 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283029 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283246 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283333 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283546 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283632 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283846 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.283932 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284141 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284236 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284435 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284522 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284753 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.284855 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285049 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285144 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285342 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285428 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285635 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285735 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.285932 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286019 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286226 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286313 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286519 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286614 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286836 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.286925 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287129 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287214 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287424 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287572 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287744 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.287874 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288060 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288190 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288362 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288511 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288696 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288798 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.288998 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289084 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289309 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289446 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289610 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289750 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.289912 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.290091 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.290247 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.290381 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.290515 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 11:56:16.290648 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.290790 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.291105 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.291406 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.291710 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.292011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.292311 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.292611 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.292938 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.293260 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.293576 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.293873 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.294186 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.294492 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.294802 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.295107 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.295407 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.295725 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.296028 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.296329 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.296631 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.296967 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.297266 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.297572 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.297885 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.298200 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.298513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.298814 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.299129 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.299443 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.299777 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.300079 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.300411 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.300756 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.301063 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.301364 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.301658 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:16.301896 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103108) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 11:56:16.347739 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO dbcounter [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Registered counter for database nova_api May 19 11:56:16.359053 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO dbcounter [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Registered counter for database nova_api May 19 11:56:16.368045 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_db.sqlalchemy.engines [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103109) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.375048 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writer thread running {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.379090 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_db.sqlalchemy.engines [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.385854 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writer thread running {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.430609 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.context [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fdac0748-340c-456f-b8b7-e62d6fcd543f(cell1) {{(pid=103109) load_cells /opt/stack/nova/nova/context.py:464}} May 19 11:56:16.436018 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 11:56:16.437600 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 11:56:16.438375 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 11:56:16.440309 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 11:56:16.441170 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.context [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fdac0748-340c-456f-b8b7-e62d6fcd543f(cell1) {{(pid=103108) load_cells /opt/stack/nova/nova/context.py:464}} May 19 11:56:16.442268 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO dbcounter [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Registered counter for database nova_cell0 May 19 11:56:16.446303 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO dbcounter [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Registered counter for database nova_cell1 May 19 11:56:16.446700 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_concurrency.lockutils [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103108) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 11:56:16.448328 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_concurrency.lockutils [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103108) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 11:56:16.449141 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_concurrency.lockutils [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103108) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 11:56:16.451063 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_concurrency.lockutils [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103108) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 11:56:16.453095 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO dbcounter [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Registered counter for database nova_cell0 May 19 11:56:16.453585 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_db.sqlalchemy.engines [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103109) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.454690 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_db.sqlalchemy.engines [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103109) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.457345 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO dbcounter [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Registered counter for database nova_cell1 May 19 11:56:16.461355 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writer thread running {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.462586 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writer thread running {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.464423 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_db.sqlalchemy.engines [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.469252 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writer thread running {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.469875 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_db.sqlalchemy.engines [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.604863 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO dbcounter [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Registered counter for database nova_cell0 May 19 11:56:16.611016 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writer thread running {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.613796 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_db.sqlalchemy.engines [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103109) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.618718 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writer thread running {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.620055 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO dbcounter [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Registered counter for database nova_cell0 May 19 11:56:16.628487 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG oslo_db.sqlalchemy.engines [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103108) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:16.633468 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writer thread running {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:16.944744 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: WARNING keystonemiddleware.auth_token [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 11:56:16.952096 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: WARNING keystonemiddleware.auth_token [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 11:56:16.960857 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: WARNING keystonemiddleware.auth_token [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 11:56:16.968223 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: WARNING keystonemiddleware.auth_token [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 11:56:17.291886 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: WARNING keystonemiddleware.auth_token [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 11:56:17.297936 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: WARNING keystonemiddleware.auth_token [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 11:56:17.303249 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b23a02c9c0 pid: 103109 (default app) May 19 11:56:17.305309 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:56:17.306473 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] 10.222.0.35 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001736 May 19 11:56:17.306892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 1/1] 10.222.0.35 () {50 vars in 788 bytes} [Thu May 19 11:56:17 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:56:17.313600 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: WARNING keystonemiddleware.auth_token [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 11:56:17.319762 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: WARNING keystonemiddleware.auth_token [None req-09a288ca-5da7-451e-a18c-b930a6646591 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 11:56:17.325026 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b23a02c9c0 pid: 103108 (default app) May 19 11:57:35.452008 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:35.453269 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 May 19 11:57:35.453759 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 2/2] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 11:57:35.467145 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_api:SELECT=3 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.468184 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_cell0:SELECT=3 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.469759 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_cell1:SELECT=3 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.470868 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.930372 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d4d55b1a-05b5-4cce-9422-cf082941bc09 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:35.961077 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d4d55b1a-05b5-4cce-9422-cf082941bc09 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.500165 May 19 11:57:35.964554 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 1/3] 10.222.0.35 () {60 vars in 1291 bytes} [Thu May 19 11:57:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:38.236200 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:38.237908 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-c6faa497-3641-4dd7-96f8-57f9ae98f96e None None] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002261 May 19 11:57:38.238461 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 3/4] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:38 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:38.303215 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-8de4b81a-4f32-43d4-a37d-9559649b91f7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "ram": 256, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 1, "id": "c1", "name": "cirros256", "os-flavor-access:is_public": true}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:38.324156 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 11:57:38.324156 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: warnings.warn(msg) May 19 11:57:38.354375 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-8de4b81a-4f32-43d4-a37d-9559649b91f7 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.108819 May 19 11:57:38.355131 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 2/5] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:38 2022] POST /compute/v2.1/flavors => generated 407 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:38.368358 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_api:SELECT=3 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.369560 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_cell0:SELECT=3 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.370564 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_cell1:SELECT=3 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.371419 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.569745 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-200ccfce-438f-40a1-97c2-e840671902ed admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:38.572325 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 11:57:38.572325 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 11:57:38.636699 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-200ccfce-438f-40a1-97c2-e840671902ed admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.273609 May 19 11:57:38.648887 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 4/6] 10.222.0.35 () {64 vars in 1335 bytes} [Thu May 19 11:57:38 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:40.863313 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-8de4b81a-4f32-43d4-a37d-9559649b91f7 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:40.865056 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-8de4b81a-4f32-43d4-a37d-9559649b91f7 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002195 May 19 11:57:40.865894 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 3/7] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:40.991721 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-565ac570-0e1f-4939-9142-a28d012ca504 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 512, "os-flavor-access:is_public": true, "name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "d1", "disk": 5, "swap": 0}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:41.009347 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 11:57:41.009347 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 11:57:41.028297 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-565ac570-0e1f-4939-9142-a28d012ca504 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.146295 May 19 11:57:41.028837 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 5/8] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:40 2022] POST /compute/v2.1/flavors => generated 404 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:41.040665 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-2a81a9ae-3913-4d5b-8e5a-a6051336805b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:41.042164 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 11:57:41.042164 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: warnings.warn(msg) May 19 11:57:41.075155 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-2a81a9ae-3913-4d5b-8e5a-a6051336805b admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038213 May 19 11:57:41.075895 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 4/9] 10.222.0.35 () {64 vars in 1335 bytes} [Thu May 19 11:57:41 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:43.328960 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-565ac570-0e1f-4939-9142-a28d012ca504 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:43.329981 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-565ac570-0e1f-4939-9142-a28d012ca504 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001583 May 19 11:57:43.330365 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 6/10] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:43.398388 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-24190881-2dc0-4aa9-b3eb-6d53a331e3d6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true, "ram": 1024, "id": "d2", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds1G"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:43.422002 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-24190881-2dc0-4aa9-b3eb-6d53a331e3d6 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.085539 May 19 11:57:43.422550 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 5/11] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:43 2022] POST /compute/v2.1/flavors => generated 404 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:43.434843 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-0fb957e0-7840-4fde-b8b3-35ff3f0af705 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:43.459424 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-0fb957e0-7840-4fde-b8b3-35ff3f0af705 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028515 May 19 11:57:43.472668 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 7/12] 10.222.0.35 () {64 vars in 1335 bytes} [Thu May 19 11:57:43 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:45.716248 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-24190881-2dc0-4aa9-b3eb-6d53a331e3d6 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:45.717454 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-24190881-2dc0-4aa9-b3eb-6d53a331e3d6 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001604 May 19 11:57:45.717825 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 6/13] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:45.791910 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-6cf9f294-db8c-4029-971f-630983640a2e admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 10, "id": "d3", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "rxtx_factor": 1.0, "vcpus": 2, "name": "ds2G"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:45.811874 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-6cf9f294-db8c-4029-971f-630983640a2e admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.087815 May 19 11:57:45.812436 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 8/14] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:45 2022] POST /compute/v2.1/flavors => generated 404 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:45.824957 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-32707fc4-aaec-40c0-a8c4-0ecdc085e5dc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:45.853160 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-32707fc4-aaec-40c0-a8c4-0ecdc085e5dc admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032082 May 19 11:57:45.853771 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 7/15] 10.222.0.35 () {64 vars in 1335 bytes} [Thu May 19 11:57:45 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:48.107307 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-6cf9f294-db8c-4029-971f-630983640a2e admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:48.108476 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-6cf9f294-db8c-4029-971f-630983640a2e admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001602 May 19 11:57:48.109233 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 9/16] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:48.174177 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "name": "ds4G", "id": "d4", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:48.194885 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.079787 May 19 11:57:48.195691 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 8/17] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:48 2022] POST /compute/v2.1/flavors => generated 404 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 11:57:48.207214 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-e496fa95-e98d-47ca-bb6a-723ac1365557 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:48.232349 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-e496fa95-e98d-47ca-bb6a-723ac1365557 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028747 May 19 11:57:48.232945 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 10/18] 10.222.0.35 () {64 vars in 1335 bytes} [Thu May 19 11:57:48 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:50.500762 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:50.502888 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002568 May 19 11:57:50.503833 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 9/19] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:50 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:50.567849 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-865be2b5-96db-4675-8123-cc1cad46428c admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:50.593921 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-865be2b5-96db-4675-8123-cc1cad46428c admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.083602 May 19 11:57:50.595992 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 11/20] 10.222.0.35 () {60 vars in 1291 bytes} [Thu May 19 11:57:50 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:57:52.912858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:52.914185 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d88a595a-0d39-4bb3-b7f8-83bc2145fddc admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001960 May 19 11:57:52.914793 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 10/21] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:52.986140 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-a6e7cfd9-cdfa-4ce0-ba58-97863e192ce0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "1", "swap": 0, "name": "m1.tiny", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 1, "vcpus": 1}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:53.005127 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-a6e7cfd9-cdfa-4ce0-ba58-97863e192ce0 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.084119 May 19 11:57:53.005674 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 12/22] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:52 2022] POST /compute/v2.1/flavors => generated 402 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:53.018751 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-51d4f9dc-9da5-4e5d-838b-f0c7ad1c983f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:53.047444 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-51d4f9dc-9da5-4e5d-838b-f0c7ad1c983f admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033412 May 19 11:57:53.048196 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 11/23] 10.222.0.35 () {64 vars in 1332 bytes} [Thu May 19 11:57:53 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:57:55.996497 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-a6e7cfd9-cdfa-4ce0-ba58-97863e192ce0 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:55.997690 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-a6e7cfd9-cdfa-4ce0-ba58-97863e192ce0 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001845 May 19 11:57:55.998138 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 13/24] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:56.059264 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-f4a191ed-4892-4c4d-a7d4-7b707f5507e9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 1, "name": "m1.small", "swap": 0, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "os-flavor-access:is_public": true, "id": "2"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:56.079862 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-f4a191ed-4892-4c4d-a7d4-7b707f5507e9 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.075753 May 19 11:57:56.080500 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 12/25] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:56 2022] POST /compute/v2.1/flavors => generated 405 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:56.096573 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-9654b6f0-3bb8-4eaf-ad6d-d921703159d3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:56.128271 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-9654b6f0-3bb8-4eaf-ad6d-d921703159d3 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035441 May 19 11:57:56.128896 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 14/26] 10.222.0.35 () {64 vars in 1332 bytes} [Thu May 19 11:57:56 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 11:57:58.404982 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-f4a191ed-4892-4c4d-a7d4-7b707f5507e9 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:57:58.405978 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-f4a191ed-4892-4c4d-a7d4-7b707f5507e9 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 May 19 11:57:58.406382 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 13/27] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:57:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:57:58.465631 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-9bcb2318-ba42-4bdc-9e51-b09076803364 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "m1.medium", "vcpus": 2, "ram": 4096, "id": "3", "os-flavor-access:is_public": true, "disk": 40, "swap": 0}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:58.484273 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-9bcb2318-ba42-4bdc-9e51-b09076803364 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.072270 May 19 11:57:58.484841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 15/28] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:57:58 2022] POST /compute/v2.1/flavors => generated 406 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:57:58.496928 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-23f66072-bf9c-44b9-8bbf-a367e81f6dfa admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:57:58.520226 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-23f66072-bf9c-44b9-8bbf-a367e81f6dfa admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027184 May 19 11:57:58.520833 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 14/29] 10.222.0.35 () {64 vars in 1332 bytes} [Thu May 19 11:57:58 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 11:58:00.745948 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-9bcb2318-ba42-4bdc-9e51-b09076803364 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:00.746996 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-9bcb2318-ba42-4bdc-9e51-b09076803364 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002096 May 19 11:58:00.747469 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 16/30] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:58:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:00.807282 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-4b94dd2b-ce03-492d-ad41-f9011b9b3abc admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "ram": 8192, "rxtx_factor": 1.0, "id": "4", "vcpus": 4}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:00.832412 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-4b94dd2b-ce03-492d-ad41-f9011b9b3abc admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.079027 May 19 11:58:00.832983 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 15/31] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:58:00 2022] POST /compute/v2.1/flavors => generated 405 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:00.844545 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-ad784288-f706-4eb3-88fc-4c4313245225 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:00.868012 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-ad784288-f706-4eb3-88fc-4c4313245225 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027101 May 19 11:58:00.868993 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 17/32] 10.222.0.35 () {64 vars in 1332 bytes} [Thu May 19 11:58:00 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:58:03.193615 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-4b94dd2b-ce03-492d-ad41-f9011b9b3abc admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:03.195215 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-4b94dd2b-ce03-492d-ad41-f9011b9b3abc admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002040 May 19 11:58:03.195611 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 16/33] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:58:03 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:03.257412 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "os-flavor-access:is_public": true, "name": "m1.xlarge", "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 8, "disk": 160, "rxtx_factor": 1.0, "ram": 16384}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:03.278418 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.076516 May 19 11:58:03.285879 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 18/34] 10.222.0.35 () {64 vars in 1282 bytes} [Thu May 19 11:58:03 2022] POST /compute/v2.1/flavors => generated 408 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:03.291099 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-844194e8-02e9-4789-b3af-76d8b9e4ee0d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:03.314222 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-844194e8-02e9-4789-b3af-76d8b9e4ee0d admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027000 May 19 11:58:03.324858 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 17/35] 10.222.0.35 () {64 vars in 1332 bytes} [Thu May 19 11:58:03 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 11:58:26.921179 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:26.922276 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001581 May 19 11:58:26.922674 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 19/36] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:58:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:26.936814 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.991835 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-5fcb38d7-e2bc-4bb9-8b9c-97fdcdfe9186 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:27.012239 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-5fcb38d7-e2bc-4bb9-8b9c-97fdcdfe9186 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.083011 May 19 11:58:27.012825 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 18/37] 10.222.0.35 () {60 vars in 1291 bytes} [Thu May 19 11:58:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:58:28.014038 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:28.015017 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-32c0f916-6562-4ff7-bdf3-f6834390b8ff admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 May 19 11:58:28.015543 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 20/38] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:28.244197 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-57f5d3f5-6a5e-49fc-b823-03fee652673d admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:28.250180 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.wsgi [None req-57f5d3f5-6a5e-49fc-b823-03fee652673d admin admin] HTTP exception thrown: Flavor 42 could not be found. May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-57f5d3f5-6a5e-49fc-b823-03fee652673d admin admin] Request method failure captured: May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept: application/json May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept-Encoding: gzip, deflate May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Connection: keep-alive May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Content-Length: 0 May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Host: 10.222.0.35 May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-Api-Version: compute 2.90 May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-System-Scope: None May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: User-Agent: python-novaclient May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Auth-Token: gAAAAABihjDkswCQZwcX2_n_PfTrd6U27ze0EFUhzaJvGC6fv_cUVETiyqxqOz35d-QpBden82RI8JZHvfhU1P1uIMaPehAomR12O_t_vXHTlhk6hZSJc8pTp4HwqJMNXk6WRLVGprFShWp7MUHWk0aK7Ymp2jwZgFoRnGWaPsID2WdaGzoWGAA May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Id: None May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Name: None May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Identity-Status: Confirmed May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Is-Admin-Project: True May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Id: default May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Name: Default May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Name: admin May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Role: member,admin,reader May 19 11:58:28.253451 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Roles: member,admin,reader May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant: admin May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Name: admin May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User: admin May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Id: default May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Name: Default May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Name: admin May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: method: > May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: exception: Flavor 42 could not be found. May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: action_args: {'id': '42'} May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:28.255011 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:28.255904 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:28.257182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-57f5d3f5-6a5e-49fc-b823-03fee652673d admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103108) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:28.258230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-57f5d3f5-6a5e-49fc-b823-03fee652673d admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.084113 May 19 11:58:28.258230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 19/39] 10.222.0.35 () {60 vars in 1190 bytes} [Thu May 19 11:58:28 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:28.264807 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-c73428c2-2f7d-425d-bca9-aa3264d3ac43 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:28.267796 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:28.287060 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-c73428c2-2f7d-425d-bca9-aa3264d3ac43 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025846 May 19 11:58:28.287601 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 21/40] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:28 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:58:28.297629 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d7f0a526-a5ed-42e1-9d34-702c60f43d35 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:28.318999 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d7f0a526-a5ed-42e1-9d34-702c60f43d35 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025011 May 19 11:58:28.319554 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 20/41] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:28 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 11:58:28.329148 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-58d76497-f4e2-4135-9c74-dc6e7b1cfaab admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:28.335536 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.wsgi [None req-58d76497-f4e2-4135-9c74-dc6e7b1cfaab admin admin] HTTP exception thrown: Flavor 42 could not be found. May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-58d76497-f4e2-4135-9c74-dc6e7b1cfaab admin admin] Request method failure captured: May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept: application/json May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept-Encoding: gzip, deflate May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Connection: keep-alive May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Content-Length: 0 May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Host: 10.222.0.35 May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-Api-Version: compute 2.90 May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-System-Scope: None May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: User-Agent: python-novaclient May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Auth-Token: gAAAAABihjDkswCQZwcX2_n_PfTrd6U27ze0EFUhzaJvGC6fv_cUVETiyqxqOz35d-QpBden82RI8JZHvfhU1P1uIMaPehAomR12O_t_vXHTlhk6hZSJc8pTp4HwqJMNXk6WRLVGprFShWp7MUHWk0aK7Ymp2jwZgFoRnGWaPsID2WdaGzoWGAA May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Id: None May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Name: None May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Identity-Status: Confirmed May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Is-Admin-Project: True May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Id: default May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Name: Default May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Name: admin May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Role: member,admin,reader May 19 11:58:28.338841 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Roles: member,admin,reader May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant: admin May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Name: admin May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User: admin May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Id: default May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Name: Default May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Name: admin May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: method: > May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: exception: Flavor 42 could not be found. May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: action_args: {'id': '42'} May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:28.340362 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:28.341342 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:28.342590 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-58d76497-f4e2-4135-9c74-dc6e7b1cfaab admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103109) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:28.343764 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-58d76497-f4e2-4135-9c74-dc6e7b1cfaab admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014673 May 19 11:58:28.343764 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 22/42] 10.222.0.35 () {60 vars in 1190 bytes} [Thu May 19 11:58:28 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:29.231173 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-d7f0a526-a5ed-42e1-9d34-702c60f43d35 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:29.232182 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-d7f0a526-a5ed-42e1-9d34-702c60f43d35 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 May 19 11:58:29.232607 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 21/43] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:29.423868 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-0e6a23d0-2cbb-43d5-a9ad-4093b5b40b43 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:29.430424 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.wsgi [None req-0e6a23d0-2cbb-43d5-a9ad-4093b5b40b43 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-0e6a23d0-2cbb-43d5-a9ad-4093b5b40b43 admin admin] Request method failure captured: May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept: application/json May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept-Encoding: gzip, deflate May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Connection: keep-alive May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Content-Length: 0 May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Host: 10.222.0.35 May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-Api-Version: compute 2.90 May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-System-Scope: None May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: User-Agent: python-novaclient May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Auth-Token: gAAAAABihjDlHcZr1Vhpqp8BlyCvqzX2s6ORdJB_NkaW6jd6eCwYoxpLX-nZS1T0OnFAIppzs6432xFlIa-60Il_uwDUY6sezg9T2ZGN6BoIOWlzYL4zzouyf7S-GAgRQcJj80nFtsGI2m1BbpYEUHhnCJ6geVQFcoxzJViZ0ZvUKMTLuuQ276A May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Id: None May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Name: None May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Identity-Status: Confirmed May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Is-Admin-Project: True May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Id: default May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Name: Default May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Name: admin May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Role: member,admin,reader May 19 11:58:29.431901 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Roles: member,admin,reader May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant: admin May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Name: admin May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User: admin May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Id: default May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Name: Default May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Name: admin May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: method: > May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: exception: Flavor 84 could not be found. May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: action_args: {'id': '84'} May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:29.433478 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:29.434596 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:29.436694 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-0e6a23d0-2cbb-43d5-a9ad-4093b5b40b43 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103109) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:29.438837 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-0e6a23d0-2cbb-43d5-a9ad-4093b5b40b43 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.066336 May 19 11:58:29.438837 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 23/44] 10.222.0.35 () {60 vars in 1190 bytes} [Thu May 19 11:58:29 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:29.445239 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1052e0db-aad7-4f53-8dc6-c218f36c6993 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:29.459671 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1052e0db-aad7-4f53-8dc6-c218f36c6993 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018022 May 19 11:58:29.460199 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 22/45] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 11:58:29.472365 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-d941e368-58dd-409c-8fa0-279ccfac9d2c admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:29.486305 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-d941e368-58dd-409c-8fa0-279ccfac9d2c admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017490 May 19 11:58:29.486829 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 24/46] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:58:29.496077 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-5df7c37b-96f2-4b10-8b2b-ef54a2254a91 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:29.501728 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.wsgi [None req-5df7c37b-96f2-4b10-8b2b-ef54a2254a91 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-5df7c37b-96f2-4b10-8b2b-ef54a2254a91 admin admin] Request method failure captured: May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept: application/json May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept-Encoding: gzip, deflate May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Connection: keep-alive May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Content-Length: 0 May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Host: 10.222.0.35 May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-Api-Version: compute 2.90 May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-System-Scope: None May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: User-Agent: python-novaclient May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Auth-Token: gAAAAABihjDlHcZr1Vhpqp8BlyCvqzX2s6ORdJB_NkaW6jd6eCwYoxpLX-nZS1T0OnFAIppzs6432xFlIa-60Il_uwDUY6sezg9T2ZGN6BoIOWlzYL4zzouyf7S-GAgRQcJj80nFtsGI2m1BbpYEUHhnCJ6geVQFcoxzJViZ0ZvUKMTLuuQ276A May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Id: None May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Name: None May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Identity-Status: Confirmed May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Is-Admin-Project: True May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Id: default May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Name: Default May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Name: admin May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Role: member,admin,reader May 19 11:58:29.503230 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Roles: member,admin,reader May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant: admin May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Name: admin May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User: admin May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Id: default May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Name: Default May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Name: admin May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: method: > May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: exception: Flavor 84 could not be found. May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: action_args: {'id': '84'} May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:29.504770 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:29.505669 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:29.507513 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-5df7c37b-96f2-4b10-8b2b-ef54a2254a91 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103108) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:29.508917 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-5df7c37b-96f2-4b10-8b2b-ef54a2254a91 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012113 May 19 11:58:29.508917 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 23/47] 10.222.0.35 () {60 vars in 1190 bytes} [Thu May 19 11:58:29 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:30.390935 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-d941e368-58dd-409c-8fa0-279ccfac9d2c admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:30.391936 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-d941e368-58dd-409c-8fa0-279ccfac9d2c admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001424 May 19 11:58:30.392322 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 25/48] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:30.580689 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-f34128a7-a72d-4bb6-88cb-ae584f52d398 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:30.590561 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.wsgi [None req-f34128a7-a72d-4bb6-88cb-ae584f52d398 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-f34128a7-a72d-4bb6-88cb-ae584f52d398 admin admin] Request method failure captured: May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept: application/json May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Accept-Encoding: gzip, deflate May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Connection: keep-alive May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Content-Length: 0 May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Host: 10.222.0.35 May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-Api-Version: compute 2.90 May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Openstack-System-Scope: None May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: User-Agent: python-novaclient May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Auth-Token: gAAAAABihjDmqVT0NkPJVZ7pYt0059inXcU-59MBZFXcRlxdrYHZuajX0zlMwyZPkuKEKSqlZ3o_WgYOdWYEE0dJVzF1oGvxE47MS5-4WF7wV_eQHYBOrQkP_6bVe06aIhExJfYM1BOwFiK9q_ttsAiWet_KsWrOjAY_Hc0vy8a2rKiXt9WseuU May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Id: None May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Domain-Name: None May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Identity-Status: Confirmed May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Is-Admin-Project: True May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Id: default May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Domain-Name: Default May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Project-Name: admin May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Role: member,admin,reader May 19 11:58:30.592566 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Roles: member,admin,reader May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant: admin May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-Tenant-Name: admin May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User: admin May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Id: default May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Domain-Name: Default May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: X-User-Name: admin May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: method: > May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: exception: Flavor 451 could not be found. May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: action_args: {'id': '451'} May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:30.594102 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:30.595078 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: ERROR nova.api.openstack.wsgi May 19 11:58:30.596481 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-f34128a7-a72d-4bb6-88cb-ae584f52d398 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103108) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:30.597720 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-f34128a7-a72d-4bb6-88cb-ae584f52d398 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.067545 May 19 11:58:30.597720 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 24/49] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 11:58:30 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:30.608878 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-b89ce48d-43f0-4d8f-88c0-b0b81341824f admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:30.631186 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-b89ce48d-43f0-4d8f-88c0-b0b81341824f admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027170 May 19 11:58:30.631860 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 26/50] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:30 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:58:30.645401 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:30.666767 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027209 May 19 11:58:30.667288 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 25/51] 10.222.0.35 () {60 vars in 1210 bytes} [Thu May 19 11:58:30 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:58:30.681030 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-b4f0faf6-34a7-48ef-95fa-c20831a54872 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:30.689435 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.wsgi [None req-b4f0faf6-34a7-48ef-95fa-c20831a54872 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-b4f0faf6-34a7-48ef-95fa-c20831a54872 admin admin] Request method failure captured: May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept: application/json May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Accept-Encoding: gzip, deflate May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Connection: keep-alive May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Content-Length: 0 May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Host: 10.222.0.35 May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-Api-Version: compute 2.90 May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Openstack-System-Scope: None May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: User-Agent: python-novaclient May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Auth-Token: gAAAAABihjDmqVT0NkPJVZ7pYt0059inXcU-59MBZFXcRlxdrYHZuajX0zlMwyZPkuKEKSqlZ3o_WgYOdWYEE0dJVzF1oGvxE47MS5-4WF7wV_eQHYBOrQkP_6bVe06aIhExJfYM1BOwFiK9q_ttsAiWet_KsWrOjAY_Hc0vy8a2rKiXt9WseuU May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Id: None May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Domain-Name: None May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Identity-Status: Confirmed May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Is-Admin-Project: True May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Openstack-Nova-Api-Version: 2.90 May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Id: default May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Domain-Name: Default May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Project-Name: admin May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Role: member,admin,reader May 19 11:58:30.691761 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Roles: member,admin,reader May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_f8b904bb5322427b8794f68069b5704c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/f8b904bb5322427b8794f68069b5704c"}]}] May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant: admin May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Id: f8b904bb5322427b8794f68069b5704c May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-Tenant-Name: admin May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User: admin May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Id: default May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Domain-Name: Default May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Id: f3118ad2395a4a5ea5565be712f02917 May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: X-User-Name: admin May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: method: > May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: exception: Flavor 451 could not be found. May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: action_args: {'id': '451'} May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 11:58:30.693673 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:30.694929 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: ERROR nova.api.openstack.wsgi May 19 11:58:30.696892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-b4f0faf6-34a7-48ef-95fa-c20831a54872 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103109) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 11:58:30.698413 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-b4f0faf6-34a7-48ef-95fa-c20831a54872 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.017960 May 19 11:58:30.698413 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 27/52] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 11:58:30 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 19 11:58:31.582735 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:31.583728 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 May 19 11:58:31.584118 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 26/53] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:31.780631 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-985324fc-08c1-4463-8f97-2730384a3374 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:31.801385 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-985324fc-08c1-4463-8f97-2730384a3374 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.084321 May 19 11:58:31.801921 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 28/54] 10.222.0.35 () {64 vars in 1235 bytes} [Thu May 19 11:58:31 2022] POST /compute/v2.1/flavors => generated 404 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:32.681076 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:32.682108 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001536 May 19 11:58:32.682495 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 27/55] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:32.883888 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-66bd3b51-96c1-4d3b-a4a5-52c5781f5ed5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:32.903987 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-66bd3b51-96c1-4d3b-a4a5-52c5781f5ed5 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.087931 May 19 11:58:32.904569 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 29/56] 10.222.0.35 () {64 vars in 1235 bytes} [Thu May 19 11:58:32 2022] POST /compute/v2.1/flavors => generated 405 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:33.791278 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:33.792308 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001512 May 19 11:58:33.792739 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 28/57] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:33.985822 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-fbeefec6-cf17-4278-86bc-7db57c7e6a0b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:34.006193 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-fbeefec6-cf17-4278-86bc-7db57c7e6a0b admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.079226 May 19 11:58:34.006732 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 30/58] 10.222.0.35 () {64 vars in 1235 bytes} [Thu May 19 11:58:33 2022] POST /compute/v2.1/flavors => generated 407 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:34.891469 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:34.892800 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001878 May 19 11:58:34.893286 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 29/59] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:35.089065 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-ff4649dc-1ac7-4b32-a2a4-96de7d36fb4e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 11:58:35.111276 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-ff4649dc-1ac7-4b32-a2a4-96de7d36fb4e admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.079320 May 19 11:58:35.111816 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 31/60] 10.222.0.35 () {64 vars in 1235 bytes} [Thu May 19 11:58:35 2022] POST /compute/v2.1/flavors => generated 409 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 11:58:36.005963 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:36.006939 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001433 May 19 11:58:36.007301 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 30/61] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 11:58:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:58:36.200141 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-21e28383-ab73-4793-b69e-4495ab347217 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:58:36.220747 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-21e28383-ab73-4793-b69e-4495ab347217 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.079314 May 19 11:58:36.221821 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 32/62] 10.222.0.35 () {60 vars in 1202 bytes} [Thu May 19 11:58:36 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:59:37.712440 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.714012 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-1cacfcd9-e38d-4f17-b48c-91c52429c128 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002162 May 19 11:59:37.714598 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 31/63] 10.222.0.35 () {54 vars in 930 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 11:59:37.722516 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:37.775962 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-ecec0258-c9f7-4aa0-8e9a-913b33c81212 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.796448 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-ecec0258-c9f7-4aa0-8e9a-913b33c81212 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.075376 May 19 11:59:37.797005 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 33/64] 10.222.0.35 () {60 vars in 1291 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 11:59:37.818414 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG dbcounter [-] [103108] Writing DB stats nova_api:SELECT=14 {{(pid=103108) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:37.824130 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-6918970c-0dd8-4d67-a941-12581ebae17d admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.836436 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-6918970c-0dd8-4d67-a941-12581ebae17d admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.020201 May 19 11:59:37.837290 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 32/65] 10.222.0.35 () {60 vars in 1282 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:59:37.850446 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-3d51a0d1-370a-41d1-aadd-0d54a4e5290d admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.858238 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-3d51a0d1-370a-41d1-aadd-0d54a4e5290d admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011426 May 19 11:59:37.858785 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 34/66] 10.222.0.35 () {60 vars in 1285 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:59:37.869939 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.878606 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012230 May 19 11:59:37.879126 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 33/67] 10.222.0.35 () {60 vars in 1285 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 11:59:37.893905 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-fc6c940e-dec0-4948-98c8-a5fda4a92c3a admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 11:59:37.907592 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-fc6c940e-dec0-4948-98c8-a5fda4a92c3a admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.020261 May 19 11:59:37.908578 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 35/68] 10.222.0.35 () {60 vars in 1282 bytes} [Thu May 19 11:59:37 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:03:19.056465 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:19.057778 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001976 May 19 12:03:19.058313 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 34/69] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 12:03:19 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:03:19.223465 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_api:SELECT=6 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:19.352354 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:19.353568 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:03:19.353568 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 12:03:19.354810 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:19.355245 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:19.356020 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:19.356441 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:19.390348 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-9455e15d-84fa-44ea-bc25-a68de95cb603 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.169717 May 19 12:03:19.390894 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 36/70] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 12:03:19 2022] GET /compute/v2.1/os-services => generated 794 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:03:20.318194 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:20.319192 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 May 19 12:03:20.319579 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 35/71] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 12:03:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:03:20.518395 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:20.519970 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:03:20.519970 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 12:03:20.521314 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:20.522088 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:20.522892 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:20.523311 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:20.542876 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-339d1421-b07d-49f5-b7a1-6b683155329c admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.087536 May 19 12:03:20.543413 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 37/72] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 12:03:20 2022] GET /compute/v2.1/os-services => generated 794 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:03:31.893797 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:31.894801 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 May 19 12:03:31.895167 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 36/73] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 12:03:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:03:32.039890 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_cell0:SELECT=4 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.040628 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_cell1:SELECT=4 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.041391 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG dbcounter [-] [103109] Writing DB stats nova_api:SELECT=8 {{(pid=103109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.097254 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:32.098430 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:03:32.098430 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 12:03:32.099679 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:32.100113 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:32.100913 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:32.101351 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:32.125710 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-a17cf730-ae0b-44cd-8023-a71dec38ca1a admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.90 time: 0.091828 May 19 12:03:32.126259 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 38/74] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 12:03:32 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:03:33.037981 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: DEBUG nova.api.openstack.wsgi [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] Calling method '>' {{(pid=103108) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:33.038985 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: INFO nova.api.openstack.requestlog [None req-70791d15-8bb8-4451-b963-187c3328accd admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 May 19 12:03:33.039382 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: [pid: 103108|app: 0|req: 37/75] 10.222.0.35 () {56 vars in 1074 bytes} [Thu May 19 12:03:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:03:33.242210 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG nova.api.openstack.wsgi [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] Calling method '>' {{(pid=103109) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:03:33.243448 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:03:33.243448 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: warnings.warn(msg) May 19 12:03:33.244689 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:33.245144 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:33.245936 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:03:33.246356 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: DEBUG oslo_concurrency.lockutils [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103109) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:03:33.268585 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: INFO nova.api.openstack.requestlog [None req-742f7cc5-dbb8-4222-afe2-dcd9f8b90db9 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.90 time: 0.090596 May 19 12:03:33.269178 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: [pid: 103109|app: 0|req: 39/76] 10.222.0.35 () {60 vars in 1193 bytes} [Thu May 19 12:03:33 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:04:30.604997 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: Thu May 19 12:04:30 2022 - graceful shutdown triggered... May 19 12:04:30.605511 dv-cinder-804035-46-iscsi devstack@n-api.service[103108]: Gracefully killing worker 1 (pid: 103108)... May 19 12:04:30.605141 dv-cinder-804035-46-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 19 12:04:30.606455 dv-cinder-804035-46-iscsi devstack@n-api.service[103109]: Gracefully killing worker 2 (pid: 103109)... May 19 12:04:31.606275 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: worker 1 buried after 1 seconds May 19 12:04:31.606275 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: worker 2 buried after 1 seconds May 19 12:04:31.606275 dv-cinder-804035-46-iscsi devstack@n-api.service[103107]: goodbye to uWSGI. May 19 12:04:31.608768 dv-cinder-804035-46-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 19 12:04:31.609686 dv-cinder-804035-46-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 19 12:04:31.613982 dv-cinder-804035-46-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 19 12:04:31.631969 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 12:04:31 2022] *** May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: nodename: dv-cinder-804035-46-iscsi May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: machine: x86_64 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: clock source: unix May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: pcre jit disabled May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: detected number of CPU cores: 4 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: current working directory: / May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: detected binary path: /usr/bin/uwsgi-core May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: your processes number limit is 47680 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: your memory page size is 4096 bytes May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: detected max file descriptor number: 2048 May 19 12:04:31.644941 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: lock engine: pthread robust mutexes May 19 12:04:31.645969 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: thunder lock: enabled May 19 12:04:31.645969 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 19 12:04:31.645969 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 12:04:31.691829 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: Python main interpreter initialized at 0x561e88d869c0 May 19 12:04:31.691829 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: python threads support enabled May 19 12:04:31.691829 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: your server socket listen backlog is limited to 100 connections May 19 12:04:31.691829 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: your mercy for graceful operations on workers is 90 seconds May 19 12:04:31.692278 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: mapped 403077 bytes (393 KB) for 2 cores May 19 12:04:31.692278 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: *** Operational MODE: preforking *** May 19 12:04:31.692537 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: *** uWSGI is running in multiple interpreter mode *** May 19 12:04:31.692537 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: spawned uWSGI master process (pid: 117519) May 19 12:04:31.692748 dv-cinder-804035-46-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 19 12:04:31.693379 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: spawned uWSGI worker 1 (pid: 117532, cores: 1) May 19 12:04:31.704997 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: spawned uWSGI worker 2 (pid: 117533, cores: 1) May 19 12:04:31.704997 dv-cinder-804035-46-iscsi devstack@n-api.service[117519]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 12:04:32.900220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 19 12:04:32.900889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 19 12:04:33.690284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:04:33.690284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: from cryptography.utils import int_from_bytes May 19 12:04:33.690632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:04:33.690632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: from cryptography.utils import int_from_bytes May 19 12:04:33.691251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:04:33.691251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: from cryptography.utils import int_from_bytes May 19 12:04:33.691568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:04:33.691568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: from cryptography.utils import int_from_bytes May 19 12:04:36.118786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 12:04:36.119520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 12:04:36.119520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 12:04:36.119905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 12:04:36.120163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 12:04:36.120545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.120873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.121209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.121536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.121850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.122154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.122468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.122786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.123106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.123355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.123665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.123986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-804035-46-iscsi {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.124295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.124584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.124914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.125025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 12:04:36.125404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 12:04:36.125493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.125493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.125737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 12:04:36.125886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.126102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 12:04:36.126205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.126357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 12:04:36.126595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.126742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.127990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.128980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.129921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-804035-46-iscsi {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-804035-46-iscsi {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.130862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.131829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.132950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.133815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.134944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.135875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-804035-46-iscsi {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.136943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.137949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.138977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.139870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.140969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.141873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.142832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.143035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.143149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.143353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.143497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.144935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.145886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.146947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.147096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.147237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.147338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.148861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.149958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.150878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.151967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.152996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.153929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.154823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.155989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.156949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.157155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.157370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.157495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.157694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.157786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.158954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.159946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.160035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.160254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.160409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.160567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.160843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.161771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.162973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.163158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.163262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.163550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.163699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.163909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.164923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.165906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.166039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.166241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.166340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.166585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.166676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.166900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.166990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.167208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.167297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.167516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.167611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.167815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.167918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.168133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.168222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.168447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.168543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.168778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.168890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.169084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.169183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.169391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.169486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.169704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.169807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:04:36.170021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.170972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.171999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.172967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.173910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.174832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.175962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.176925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.177937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.178885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.179846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.180996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.181879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.182791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.183957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.184994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.185890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.186897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.187983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.188938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.189843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.190018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.190132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.190317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.190432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.191662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.191853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.191853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.192726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.194392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.194495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.194591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.194591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.194884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.195852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.196922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.197943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.198956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.199863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.200841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.201976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.202906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.203936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.204899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.205876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.206940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.207954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.208909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.209836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.210861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.211994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.212925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.213896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.214842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.215868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.216039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.216148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.216351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.219562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.219562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.219562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.219848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.219934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.220945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.221983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.222956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.223981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.224868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.225138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.225239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.225441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.225671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.225771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.226928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.227970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.228944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.229918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.230852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.231954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.232966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.233889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.234963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.235944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.236323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.236493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.236666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.236847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.237138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.237329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.237547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.237729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.237966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.238972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.239123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.239388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.239594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.239746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.239895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.240867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.241874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.242929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.243995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.244940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.245869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.246026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.246175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.246497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.246654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.246804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.247032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.247032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.247477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.247477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.247883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.248130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.248378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.248537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.248780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.248948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.249990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.250783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.251996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.252938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.253862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.254872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.255955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.256111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.256282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.256459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.256824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.256985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.257142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.257471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.260303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.260911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.261928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.263296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.263296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.263604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.263757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.263917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.264174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.264402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.264663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.264913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.265156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.265419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.265679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.265941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.266208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.266458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.267005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.267005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.267691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.267942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.268201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.268457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.269276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.269935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.270243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.270639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.270870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.271089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.271314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.271532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.271756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.271969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.272196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.272424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.272669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.272972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.273173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.273507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.273724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.274073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.274298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.274298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.274704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.274912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.275107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.275293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.275489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.275701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.275908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.276116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.276309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.276514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.277320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.277517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.277776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.277936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.278146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.278395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.278645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.278808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.278993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.279176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.279421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.279577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.280064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.280257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.280418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.280597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.280987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.281940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.282891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.283053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.283919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.284934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.285874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.286900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.287263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.287427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.287552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.287552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.287786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.288072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.288202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.288486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.288717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.288891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.289064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.289234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.289456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.289733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.290961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.291744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.292981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.293926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.294961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.295769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.296898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.297860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.298997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.299889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.300118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.300381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.300513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.300749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.300883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.301785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.302960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.303951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.304288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.304472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.304605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.304752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.304880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.305801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.306997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.307909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.308865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.309796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.310884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.311887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.312966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.313824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.314231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.314372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.315491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.315633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.315769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.315974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.315974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.316936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.317795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.318035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.318171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.318300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.318556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.318556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.320462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.320855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.320855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.321131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.322935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.323063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.323191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.323319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.323454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.325643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.325643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.325911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.326969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.327750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.328789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.329671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.330058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.330402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.330632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.330879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.331884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.332828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.333958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.334908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.335694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.335826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.335983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.335983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.336992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.337765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.338802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.339932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.340067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.340205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.340455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.340455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.341995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.342904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.343884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.344927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.345906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.346274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.346494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.346709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.346841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.347998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.348130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.348413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.348627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.348799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.349946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.350129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.350261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.350551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.350716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.351791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.352880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.353036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.353273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.353406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.353649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.353884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.354025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.354297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.354443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.354655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.354800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.355034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.355166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.355391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.355577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.355856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.356005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.356347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.356487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.356750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.356897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.357071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.357247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.357410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.357687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.357831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.358980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.359987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.360169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.360306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.360475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.360796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.360968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.361251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.361529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.361662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.361800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.361930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.362992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.363192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.363353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.363500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.363816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.363945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.364129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.364404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.364554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.364759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.364891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.365068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.365300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.365439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.365665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.365807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.366027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.366175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.366358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.366641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.366790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.367946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.368086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.368373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.368513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.368698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.368919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.369066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.369359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.369498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.369636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.369931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.370119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.370260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.370553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.370690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.370904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.371052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.371205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.371632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.371783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.371783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.372026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.372154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.372380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.372575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.372760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.373917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.374164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.374326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.374595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.374780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.375204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.375521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.375836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.376951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.377993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.378911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.379929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.380853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.381016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.381235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.381558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.381921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.382888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.383865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.384885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.385938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.386774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.387937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.388847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.389882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.390312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.390455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.390602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.390753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.390902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.391059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.391219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.391435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.391677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.392980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.393124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.393279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.393421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.393916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.394805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.395924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.396929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.397082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.397235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 12:04:36.397433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.397433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.397717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.397888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.398215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.398533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.398847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.399256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.399594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.399925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.400207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.400516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.400854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.401163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.401480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.401791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.402198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.402399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.402720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.403033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.403352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.403665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.403974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.405342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.405673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.406056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.406412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.406717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.407110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.407753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.407753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.408073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.408426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.408835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.409165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.409496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.409913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.410313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.410624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.410953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.411382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.411695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.411997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:04:36.412392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 12:04:36.456390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO dbcounter [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Registered counter for database nova_api May 19 12:04:36.472988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO dbcounter [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Registered counter for database nova_api May 19 12:04:36.477065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_db.sqlalchemy.engines [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.484536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writer thread running {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.493417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_db.sqlalchemy.engines [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.501318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writer thread running {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.540155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.context [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fdac0748-340c-456f-b8b7-e62d6fcd543f(cell1) {{(pid=117533) load_cells /opt/stack/nova/nova/context.py:464}} May 19 12:04:36.544575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:04:36.546157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:04:36.546919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:04:36.548887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:04:36.550852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO dbcounter [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Registered counter for database nova_cell0 May 19 12:04:36.554861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO dbcounter [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Registered counter for database nova_cell1 May 19 12:04:36.558775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.context [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fdac0748-340c-456f-b8b7-e62d6fcd543f(cell1) {{(pid=117532) load_cells /opt/stack/nova/nova/context.py:464}} May 19 12:04:36.561987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_db.sqlalchemy.engines [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.563457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:04:36.565158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_db.sqlalchemy.engines [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.565663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:04:36.566822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:04:36.569972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:04:36.571207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writer thread running {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.572019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO dbcounter [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Registered counter for database nova_cell0 May 19 12:04:36.576249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO dbcounter [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Registered counter for database nova_cell1 May 19 12:04:36.583589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_db.sqlalchemy.engines [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.586007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_db.sqlalchemy.engines [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.590568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writer thread running {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.708817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writer thread running {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.719273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO dbcounter [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Registered counter for database nova_cell0 May 19 12:04:36.726627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_db.sqlalchemy.engines [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.728113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writer thread running {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.731881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writer thread running {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:36.738447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO dbcounter [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Registered counter for database nova_cell0 May 19 12:04:36.745400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_db.sqlalchemy.engines [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:04:36.750080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writer thread running {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:04:37.047280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: WARNING keystonemiddleware.auth_token [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 12:04:37.055661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: WARNING keystonemiddleware.auth_token [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 12:04:37.075392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: WARNING keystonemiddleware.auth_token [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 12:04:37.083293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: WARNING keystonemiddleware.auth_token [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 12:04:37.401205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: WARNING keystonemiddleware.auth_token [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 12:04:37.407410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: WARNING keystonemiddleware.auth_token [None req-c4f286e3-aa54-4c1d-9521-1391c02e09c8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 12:04:37.412685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561e88d869c0 pid: 117533 (default app) May 19 12:04:37.428920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: WARNING keystonemiddleware.auth_token [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 19 12:04:37.435153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: WARNING keystonemiddleware.auth_token [None req-3953a3dc-0f61-4402-bca1-7033b398e152 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 19 12:04:37.440432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561e88d869c0 pid: 117532 (default app) May 19 12:05:18.555578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:18.556491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:18.557768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:18.559325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:20.249603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-51153be4-9a49-473d-b84c-aa0c03d5c9e0 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Action: 'create', calling method: .version_select at 0x7fe20e0b5550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1481780329", "metadata": {"Type": "work"}}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:22.676418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-51153be4-9a49-473d-b84c-aa0c03d5c9e0 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 4.136193 May 19 12:05:22.680831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1/1] 10.222.0.35 () {60 vars in 1155 bytes} [Thu May 19 12:05:18 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 4142 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) May 19 12:05:22.692080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:22.693089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:22.702127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:22.704966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:23.063215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1688409715", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "networks": [{"uuid": "551fdda8-b2ff-46db-b7db-7352aec22008"}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:23.344189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] validate_networks() for [('551fdda8-b2ff-46db-b7db-7352aec22008', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:05:23.410457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0efe1a2b-c89e-48e5-a994-47ace8b00a34 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20e082040>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:23.958452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0efe1a2b-c89e-48e5-a994-47ace8b00a34 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 200 len: 353 microversion: 2.1 time: 1.272002 May 19 12:05:23.960057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1/2] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:05:22 2022] GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 353 bytes in 1276 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:24.972841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f1c68772-6f66-4c5f-9c06-3221e41b0c1b tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20dfce4c0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:25.081428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f1c68772-6f66-4c5f-9c06-3221e41b0c1b tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 200 len: 354 microversion: 2.1 time: 0.113450 May 19 12:05:25.089085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 2/3] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:05:24 2022] GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 354 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:05:25.095652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7032f951-de4e-4e38-96d7-e6f7ee6fdec6 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20dfceb80>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:25.191357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7032f951-de4e-4e38-96d7-e6f7ee6fdec6 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 200 len: 354 microversion: 2.1 time: 0.100248 May 19 12:05:25.192094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 3/4] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:05:25 2022] GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 354 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:25.216966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-87a78277-287f-46fa-a95e-80acef9c7596 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20dfced30>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:25.404697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-87a78277-287f-46fa-a95e-80acef9c7596 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 202 len: 0 microversion: 2.1 time: 0.197942 May 19 12:05:25.408626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 4/5] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:05:25 2022] DELETE /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 0 bytes in 200 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:05:25.414638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c78a401a-6b8c-4c9b-84bd-fe6c419cd001 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20dfce550>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:25.473140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:05:25.474332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:05:25.475139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:05:25.475699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:05:25.476240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:05:25.522986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:25.523934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:25.524886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:25.525473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:25.531251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c78a401a-6b8c-4c9b-84bd-fe6c419cd001 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 200 len: 353 microversion: 2.1 time: 0.121263 May 19 12:05:25.531830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 5/6] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:05:25 2022] GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 353 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:25.594159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Getting quotas for project bb8c6f8383ec43868ace8748bdd3946b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:25.613326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Getting quotas for user 3fc0a97d39d04020b8f933dbd5a57933 and project bb8c6f8383ec43868ace8748bdd3946b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:25.630401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:05:25.633521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:05:25.763236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-88d8dfb1-a9ec-4393-bca5-9dd50dede6b7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.837061 May 19 12:05:25.765422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 2/7] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:05:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 2839 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:05:25.777393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b13c9e32-0688-4689-bc19-3c12f92ffa82 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:25.781218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b13c9e32-0688-4689-bc19-3c12f92ffa82 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:26.107120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b13c9e32-0688-4689-bc19-3c12f92ffa82 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:26.179455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b13c9e32-0688-4689-bc19-3c12f92ffa82 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1215 microversion: 2.1 time: 0.406076 May 19 12:05:26.183109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 6/8] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:25 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1215 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:26.545359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f4ea401-c0b0-47e2-bc51-620e46f88956 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Calling method '.version_select at 0x7fe20ce27040>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:26.611401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-4f4ea401-c0b0-47e2-bc51-620e46f88956 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] HTTP exception thrown: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f4ea401-c0b0-47e2-bc51-620e46f88956 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Request method failure captured: May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 HTTP/1.1 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjJ-NjdxD6UlNf_BSxlRmbhWg5eiZZ9Rrozbk00v6aL4X4Fv_SB-5FKSIa8rT9VQO7r-wvWFB1oFtQ2XsC9TeU78r71ckmxFIlmly2eH2STIu_1qfB3_rAtw31I9Bh8x3s0z44C8YP8qWYN98vkKL4sI8f13tGMapQKyHQv0yAxm24k May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: c6a2e937cae64681a70b82928e4c6928 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesGetTestJSON-1807971671 May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:05:26.616139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c6a2e937cae64681a70b82928e4c6928"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c6a2e937cae64681a70b82928e4c6928"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c6a2e937cae64681a70b82928e4c6928"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c6a2e937cae64681a70b82928e4c6928"}]}] May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesGetTestJSON-1807971671 May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: c6a2e937cae64681a70b82928e4c6928 May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesGetTestJSON-1807971671 May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesGetTestJSON-1807971671-project May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 7d110533a750491f9ff4e4834b25563a May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesGetTestJSON-1807971671-project May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '43270f7f-0cf6-4c6e-87c4-025018d94618'} May 19 12:05:26.617255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. (HTTP 404) (Request-ID: req-d83edd42-2248-4944-9791-6cb7eae0cded) May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:05:26.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:05:26.618702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 19 12:05:26.620322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f4ea401-c0b0-47e2-bc51-620e46f88956 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] Returning 404 to user: Volume 43270f7f-0cf6-4c6e-87c4-025018d94618 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f4ea401-c0b0-47e2-bc51-620e46f88956 tempest-VolumesGetTestJSON-1807971671 tempest-VolumesGetTestJSON-1807971671-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618" status: 404 len: 109 microversion: 2.1 time: 0.077738 May 19 12:05:26.621262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 3/9] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:05:26 2022] GET /compute/v2.1/os-volumes/43270f7f-0cf6-4c6e-87c4-025018d94618 => generated 109 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:05:27.199779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-985102c3-6d84-4b05-bbd3-9085cce2b204 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:27.201228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-985102c3-6d84-4b05-bbd3-9085cce2b204 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:27.666633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-985102c3-6d84-4b05-bbd3-9085cce2b204 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:27.682621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-985102c3-6d84-4b05-bbd3-9085cce2b204 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1215 microversion: 2.1 time: 0.486718 May 19 12:05:27.686997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 7/10] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:27 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1215 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:28.699581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:28.700359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:28.747070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:28.749764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:28.755486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:28.761001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:29.320762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:29.350575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d275d7a6-9544-4f8f-bd35-2a4c87db60c7 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1219 microversion: 2.1 time: 0.652774 May 19 12:05:29.352764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 4/11] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:28 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1219 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:30.074121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-306906001", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:30.311095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] validate_networks() for [('3c40f460-44ee-4118-b5ce-daffc19a643a', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:05:30.374076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:30.375533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:30.390171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:30.391284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:30.392931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:30.393523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:30.582446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:30.594616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b080c538-3134-4da8-b103-4f875e3d42ba tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1211 microversion: 2.1 time: 0.230953 May 19 12:05:30.598971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 5/12] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:30 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1211 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:31.617480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:31.618219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:31.628368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:31.629182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:31.630161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:31.630587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:31.832962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:31.845049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0b6f14e5-a7ea-40cf-83b7-e1a1b34d28af tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1211 microversion: 2.1 time: 0.231412 May 19 12:05:31.847413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 6/13] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:31 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1211 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:32.867206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:32.867821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:32.885162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:32.886188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:32.888689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:32.889350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:33.086664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:05:33.087164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:05:33.087628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:05:33.088116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:05:33.088590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:05:33.117824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:33.124963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:33.126017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:33.126533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:33.209346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:33.231189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a215be23-e0b1-4e90-9808-59961a45594c tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1267 microversion: 2.1 time: 0.365667 May 19 12:05:33.232056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 7/14] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:32 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1267 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:33.236513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:33.273124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for user 36415049be5d4238a0275db21a8f378a and project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:33.314525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:05:33.323683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:05:33.518790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-09d58de9-bce8-4b3c-96e2-4f90e5819c14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.999062 May 19 12:05:33.519816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 8/15] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:05:29 2022] POST /compute/v2.1/servers => generated 383 bytes in 4001 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:05:33.530950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1a15126a-6a16-4fd3-ac12-3bc7368c9214 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:33.531602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1a15126a-6a16-4fd3-ac12-3bc7368c9214 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:33.855819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4b7a48ab-0917-4fee-b88d-b487d3351940 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: .version_select at 0x7fe20d63a3a0>, body: {"keypair": {"name": "tempest-keypair-973024615", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO1sxKlneQ6UJV0nNGljGY/ybK9obthYEVEL17A5rPA7QI5xxprh2Updaqtf/dg185+vaFTuNCrV1UQHLdT38o1mmPiK42XhKnXZbCWh8gJ5fwiR4ZhHhmc6uh7Gm2inOg=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:33.913055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-4b7a48ab-0917-4fee-b88d-b487d3351940 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:33.938219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-4b7a48ab-0917-4fee-b88d-b487d3351940 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for user eefde2311ec64430a6ed161238024df4 and project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:34.057191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4b7a48ab-0917-4fee-b88d-b487d3351940 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.459169 May 19 12:05:34.058094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 9/16] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:05:33 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:34.249048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:34.249844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:34.268794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:34.272400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:34.274622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:34.275469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:34.278892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a15126a-6a16-4fd3-ac12-3bc7368c9214 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:34.296231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1a15126a-6a16-4fd3-ac12-3bc7368c9214 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1221 microversion: 2.1 time: 0.769428 May 19 12:05:34.297395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 8/17] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:33 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1221 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:34.534542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-988946821", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:34.722807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] validate_networks() for [('3c40f460-44ee-4118-b5ce-daffc19a643a', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:05:34.758127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:34.786118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-63933d38-547f-4ab8-a95e-2b3bb33dea46 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1273 microversion: 2.1 time: 0.541211 May 19 12:05:34.786856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 10/18] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:34 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1273 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:35.566770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:35.806395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:35.807811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:35.825854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:35.826314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:35.827098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:35.827534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:37.943015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:37.970560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-35d14874-2f76-43e0-ac96-9f5dea999dde tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 2.159762 May 19 12:05:37.976924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 11/19] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:35 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 2175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:38.387300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:05:38.387300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:05:38.387855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:05:38.388247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:05:38.388786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:05:38.414107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:38.416203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:38.422076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:38.422817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:38.497676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:38.505420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for user 36415049be5d4238a0275db21a8f378a and project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:38.528116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:05:38.536122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:05:38.610311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-12dfaa69-55eb-4c3f-898e-7a861c18ad2f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.081498 May 19 12:05:38.611784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 9/20] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:05:34 2022] POST /compute/v2.1/servers => generated 383 bytes in 4084 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:05:38.627242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ed7b5ac9-2ae1-4f8a-99c8-bd3f454454ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:38.631826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ed7b5ac9-2ae1-4f8a-99c8-bd3f454454ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:38.944467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ed7b5ac9-2ae1-4f8a-99c8-bd3f454454ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:38.959830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ed7b5ac9-2ae1-4f8a-99c8-bd3f454454ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1221 microversion: 2.1 time: 0.342219 May 19 12:05:38.971099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 12/21] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:38 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1221 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:38.989978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:38.991216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:39.010426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:39.012352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:39.014005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:39.015842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:39.978888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:39.979695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:40.005098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:40.005820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:40.008535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:40.009323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:40.185938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:40.200160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5eb79695-5677-4247-a04c-4202e91f0dcd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1225 microversion: 2.1 time: 0.223014 May 19 12:05:40.201680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 13/22] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:39 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1225 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:40.635421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:40.646688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b42937b4-c42e-4bb8-9f29-e29d061a43c9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 1.662998 May 19 12:05:40.659357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 10/23] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:38 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 1676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:41.221874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:41.221874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:41.257485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:41.259581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:41.276812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:41.283790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:41.667711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:41.668402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:41.695477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:41.696981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:41.697814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:41.700193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:42.544342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:42.563531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-12712f94-d5cb-4216-92b5-7131f57ee6ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1281 microversion: 2.1 time: 1.347842 May 19 12:05:42.564193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 14/24] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:41 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1281 bytes in 1351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:43.172471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:43.227810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:43.243237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d1afb9ee-e08e-4413-a92c-c61dd6dec92a tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 1.579834 May 19 12:05:43.250801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 11/25] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:41 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 1589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:43.584495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:43.586123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:43.618687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:43.624197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:43.630145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:43.630690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:43.688628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-014c74f0-d5dd-4114-a3f6-e0335f389238 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'create', calling method: .version_select at 0x7fe20c5743a0>, body: {"keypair": {"name": "tempest-keypair-2125010663", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJB7cXj9z1vkugJ1tHS/auaS1TcuhftC6YnGXihOJB4vCwdfmI0UzHpfZ/5vJPN6H+xjQ5S5L1pSYgKbyPTO1tfbNmD5scuO1XD+Pd68ThTtGpF1tKHBlwjMwZGghv9Psg=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:43.772925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-014c74f0-d5dd-4114-a3f6-e0335f389238 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:43.794408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-014c74f0-d5dd-4114-a3f6-e0335f389238 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for user b4113cb440764819996cfffec45e0b76 and project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:43.908081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-014c74f0-d5dd-4114-a3f6-e0335f389238 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.956033 May 19 12:05:43.909658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 15/26] 10.222.0.35 () {62 vars in 1199 bytes} [Thu May 19 12:05:42 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 963 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 19 12:05:44.273658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:44.275111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:44.297798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:44.306811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:44.310879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:44.312361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:44.385099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:44.404906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4c304769-cb47-4485-8182-e4e9065b6121 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1279 microversion: 2.1 time: 0.826864 May 19 12:05:44.412529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 12/27] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:43 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1279 bytes in 837 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:45.235761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:45.247629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3e6373b7-2332-45e7-887b-120c8d8e8247 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.978389 May 19 12:05:45.249627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 16/28] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:44 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 982 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:45.427165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:45.427782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:45.447049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:45.460196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:45.463983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:45.465637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:45.732731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:45.747626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-823f7151-2d7a-44ce-9489-0522728eb132 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1279 microversion: 2.1 time: 0.324758 May 19 12:05:45.764082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 13/29] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:45 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1279 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:46.266012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:46.267496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:46.279072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.279702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.280661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.282870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.533707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-48b5a94a-fd59-476c-9f92-0fa9d2ab5bd8 req-9b3ca29a-ee05-4591-9e02-6e946a8fb1ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7fafb49c-cc35-4a08-b5e5-70871f75cc68", "tag": "3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:46.584456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-48b5a94a-fd59-476c-9f92-0fa9d2ab5bd8 req-9b3ca29a-ee05-4591-9e02-6e946a8fb1ad service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.585655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-48b5a94a-fd59-476c-9f92-0fa9d2ab5bd8 req-9b3ca29a-ee05-4591-9e02-6e946a8fb1ad service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.645605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-48b5a94a-fd59-476c-9f92-0fa9d2ab5bd8 req-9b3ca29a-ee05-4591-9e02-6e946a8fb1ad service nova] Creating event network-changed:3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4 for instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 on hv-cinder-80403 May 19 12:05:46.646150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-48b5a94a-fd59-476c-9f92-0fa9d2ab5bd8 req-9b3ca29a-ee05-4591-9e02-6e946a8fb1ad service nova] Acquired lock "compute-rpcapi-router" {{(pid=117533) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 19 12:05:46.655813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.656268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.657193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.657752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.680688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.compute.rpcapi [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 19 12:05:46.681710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] Releasing lock "compute-rpcapi-router" {{(pid=117533) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 19 12:05:46.714691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c6047d5-242e-4aa4-8ace-8813e9983ba7 None None] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.423606 May 19 12:05:46.715630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 14/30] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:05:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:46.771427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:46.772367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:46.784269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.785000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:46.786078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:46.787813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:47.074070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:47.089134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c9566b88-03e1-4bf9-bd05-acb62882a7f5 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.827259 May 19 12:05:47.101741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 17/31] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:46 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:47.131976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1568883839", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "887eaf77-1004-4cec-8309-32f7b4c05c89"}], "security_groups": [{"name": "tempest-securitygroup--1002750830"}], "key_name": "tempest-keypair-973024615", "user_data": "}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:47.370573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:47.383404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cc68fd5d-6615-45b3-80b5-269cc52256d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1279 microversion: 2.1 time: 0.616144 May 19 12:05:47.410858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 15/32] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:46 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1279 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:48.118498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:48.119115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:48.139185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:48.139634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:48.154486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:48.155067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:48.173651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] validate_networks() for [('887eaf77-1004-4cec-8309-32f7b4c05c89', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:05:48.518625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=14 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:49.274949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:49.278460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=67 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:49.299739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5d3a1ca5-b281-473a-9947-0d1eedacf5a9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 1.184959 May 19 12:05:49.303972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 16/33] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:48 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 1190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:49.314497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:49.320303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:49.339907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:49.344519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:49.345265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:49.346216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:49.346798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:50.565972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:05:50.567357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:05:50.572464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:05:50.573062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:05:50.573461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:05:50.590736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:50.592135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:50.593323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:50.596052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:50.664987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:50.673116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for user eefde2311ec64430a6ed161238024df4 and project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:50.694651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:05:50.694651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:05:50.797554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-74411adf-ee8b-40bd-a8b3-5c11cadca1ed tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.673403 May 19 12:05:50.799512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 18/34] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:05:47 2022] POST /compute/v2.1/servers => generated 402 bytes in 3676 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:05:51.094249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-0308baf4-4ff6-4a4c-a9d1-b260bfc6b2d8 req-144cc015-ff2c-48e4-97a3-17d07cec3f6e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da2ea452-9604-40b7-b9cb-65be5fa21be5", "tag": "c5bf0325-0a37-4c7a-a88a-f713bf437054"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:51.137932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:51.138457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0308baf4-4ff6-4a4c-a9d1-b260bfc6b2d8 req-144cc015-ff2c-48e4-97a3-17d07cec3f6e service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:51.138704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0308baf4-4ff6-4a4c-a9d1-b260bfc6b2d8 req-144cc015-ff2c-48e4-97a3-17d07cec3f6e service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:51.162447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2cf94cba-bc71-4693-bf89-df7bcd6f1fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 1.850461 May 19 12:05:51.163518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 17/35] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:49 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 1859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:51.185629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:51.188509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:51.201302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:51.204555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:51.205667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:51.213637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-0308baf4-4ff6-4a4c-a9d1-b260bfc6b2d8 req-144cc015-ff2c-48e4-97a3-17d07cec3f6e service nova] Creating event network-changed:c5bf0325-0a37-4c7a-a88a-f713bf437054 for instance da2ea452-9604-40b7-b9cb-65be5fa21be5 on hv-cinder-80403 May 19 12:05:51.214215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0308baf4-4ff6-4a4c-a9d1-b260bfc6b2d8 req-144cc015-ff2c-48e4-97a3-17d07cec3f6e service nova] Acquired lock "compute-rpcapi-router" {{(pid=117532) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 19 12:05:51.215034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:51.215400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:51.215845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:51.224979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:51.225959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:51.259524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.compute.rpcapi [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 19 12:05:51.266431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] Releasing lock "compute-rpcapi-router" {{(pid=117532) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 19 12:05:51.315091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-54d073d5-8fc7-41db-9edd-7d9708d6f822 None None] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.514034 May 19 12:05:51.315702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 19/36] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:05:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:05:51.332884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-857a7dc5-046e-4057-a305-9cc9dcf7837d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:51.335429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-857a7dc5-046e-4057-a305-9cc9dcf7837d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:51.496236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-857a7dc5-046e-4057-a305-9cc9dcf7837d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:51.507997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-857a7dc5-046e-4057-a305-9cc9dcf7837d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1237 microversion: 2.1 time: 0.188536 May 19 12:05:51.509756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 20/37] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1237 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:52.043343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-08160f8b-757d-4616-83f4-5723565fd7c4 req-e4598d9c-d1a9-4738-8e8a-b3f4fb759cc8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a9117bef-13c8-47cc-bb42-ea7a47d5025a", "tag": "fc1f75e7-0085-4916-9009-12d6f71197eb"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:52.059001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-08160f8b-757d-4616-83f4-5723565fd7c4 req-e4598d9c-d1a9-4738-8e8a-b3f4fb759cc8 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:52.059720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-08160f8b-757d-4616-83f4-5723565fd7c4 req-e4598d9c-d1a9-4738-8e8a-b3f4fb759cc8 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:52.073805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-08160f8b-757d-4616-83f4-5723565fd7c4 req-e4598d9c-d1a9-4738-8e8a-b3f4fb759cc8 service nova] Creating event network-changed:fc1f75e7-0085-4916-9009-12d6f71197eb for instance a9117bef-13c8-47cc-bb42-ea7a47d5025a on hv-cinder-80403 May 19 12:05:52.078288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-08160f8b-757d-4616-83f4-5723565fd7c4 req-e4598d9c-d1a9-4738-8e8a-b3f4fb759cc8 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044070 May 19 12:05:52.081923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 21/38] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:05:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:05:52.184536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:52.185120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:52.193501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:52.193960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:52.194860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:52.197598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:52.205261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:52.218704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c77f61da-0956-4b68-9d84-a0ab94162059 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 1.040100 May 19 12:05:52.219392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 18/39] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:51 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 1055 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:52.374747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:52.383658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c5a6a71-5872-4e81-8399-6b65c5ac815c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.207637 May 19 12:05:52.394199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 22/40] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:52 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:52.530264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:52.531050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:52.538705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:52.539418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:52.540573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:52.552026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:52.678624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:52.691109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e524a010-0d5f-4d94-bc41-bf78c014f294 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1241 microversion: 2.1 time: 0.167661 May 19 12:05:52.691710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 19/41] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1241 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:52.961939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2048112840", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e6e51914-d344-4c2c-9d9a-e967c10c1ab5"}], "security_groups": [{"name": "tempest-securitygroup--1720457263"}], "key_name": "tempest-keypair-2125010663", "user_data": "}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:05:53.118816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] validate_networks() for [('e6e51914-d344-4c2c-9d9a-e967c10c1ab5', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:05:53.237417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:53.238265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:53.244990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.245602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.246404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.247065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.450940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.459857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bc6420f6-8ca2-42cd-9082-46354bc16a19 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.227195 May 19 12:05:53.460873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 20/42] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:53 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:53.479287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:53.480129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:53.489472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.490155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.491058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.491775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.650038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.658366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0b26c3d3-0d88-4fcd-bd7e-e30eade91e25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.184262 May 19 12:05:53.661048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 21/43] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:53 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:53.679029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:05:53.679029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:05:53.679448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:05:53.680109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:05:53.681054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:05:53.683393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=8 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:53.692335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=29 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:53.697126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.698014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.698974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.699365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.708389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:53.709273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:53.717789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.718229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.719053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:53.719413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:53.725023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=53 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:53.739824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:05:53.744348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for user b4113cb440764819996cfffec45e0b76 and project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:05:53.756133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:05:53.758766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:05:53.813852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-adb2fa97-faa7-43cb-acda-94511028479b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.856650 May 19 12:05:53.815445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 23/44] 10.222.0.35 () {62 vars in 1187 bytes} [Thu May 19 12:05:52 2022] POST /compute/v2.1/servers => generated 402 bytes in 859 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 19 12:05:53.825421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:53.826370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:53.830786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.839325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0bbbda24-44f8-43c4-8142-f5dc43b256e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1241 microversion: 2.1 time: 0.135461 May 19 12:05:53.850675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 22/45] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1241 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:53.890226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.900155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.902848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:53.904126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8d4061a1-dafd-48a9-9baa-2b2376ef1c8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1282 microversion: 2.20 time: 0.082647 May 19 12:05:53.904714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 24/46] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1282 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:05:54.477741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:54.478575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:54.485218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.485667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.486496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.486994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.649824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:54.658835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-da5c1b62-4b8a-4e04-8184-18f4e53d4fe9 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.185182 May 19 12:05:54.670382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 23/47] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:54 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:54.678089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:54.678782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:54.686977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.687420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.688179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.688732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.838113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:54.846598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9f20264f-c2f3-4b7a-9e5c-9f39bd47f6d1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.172782 May 19 12:05:54.847574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 25/48] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:54 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:54.858050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:54.858654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:54.864920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.865366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.866348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.866926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.922885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:54.923498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:54.935211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.935769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.936673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:54.937229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:54.943793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:54.951458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-915fb77b-b288-4534-947b-80482052f180 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1241 microversion: 2.1 time: 0.097207 May 19 12:05:54.952037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 24/49] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1241 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:55.008451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:55.018021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:55.019588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:55.020768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-059bb4ea-9495-44cf-8358-6b1c14c71cd6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1286 microversion: 2.20 time: 0.102033 May 19 12:05:55.021327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 26/50] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1286 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:05:55.680018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:55.681862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:55.689507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.690045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:55.696137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.696137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:55.867362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:55.873568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:55.876211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:55.903892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c76512de-6e5a-48a0-8e43-bb49b26486f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.225847 May 19 12:05:55.908960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.912264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:55.913089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.913654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:55.913876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 25/51] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:55 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:55.971729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:55.972439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:55.979313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.979753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:55.980511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:55.980965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:56.098708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:56.110979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-045a1994-09d6-4c29-a5e2-8c1e339e1d45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1241 microversion: 2.1 time: 0.146337 May 19 12:05:56.111926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 26/52] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1241 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:56.116955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:56.117546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:56.124957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:56.125579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:56.126336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:56.126894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:56.193397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:56.203820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62af094f-26a3-4524-b6e2-d7fa26f52e5f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.343980 May 19 12:05:56.204422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 27/53] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:55 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:56.261548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:56.270844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:56.272737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:56.274015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f0bc2684-a4a7-4a38-b02d-c9b3689586a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1286 microversion: 2.20 time: 0.161043 May 19 12:05:56.284519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 27/54] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1286 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:05:56.922854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:56.923448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:56.932921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:56.932921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:56.932921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:56.932921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.075866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.083972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6bb0cfe3-a07b-4d2b-9898-e0d1d88fb5c1 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.165431 May 19 12:05:57.084565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 28/55] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:56 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:57.127933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:57.129487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:57.131620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:57.138415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.138415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.138415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.138650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.221076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:57.221681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:57.230684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.233143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.233586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.234354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.234792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.239284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0350e400-96f2-431b-9694-176a1e7fa36a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1241 microversion: 2.1 time: 0.115115 May 19 12:05:57.239899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 28/56] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1241 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:57.293459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:57.294777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:57.304542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.309244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.310471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:57.313011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:57.421189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.425185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.434376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-978840ce-6df1-4f1f-85b5-2ff53255c2e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.217047 May 19 12:05:57.437311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.440294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:57.441846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-290d90dd-4511-4cfc-9b65-cdb6f40d1dbe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1286 microversion: 2.20 time: 0.153693 May 19 12:05:57.442733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 29/57] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1286 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:05:57.446046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 29/58] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:57 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:05:58.102610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:58.103786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:58.110926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.111724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.112904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.113804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.257425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:58.258925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:58.279419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.281331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.286380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.290243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.327616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.338332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6faba501-6ba8-4506-a4d4-780d014289cd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.239861 May 19 12:05:58.342168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 30/59] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:58 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:58.430493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.444877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8dd030d6-cc2b-4a2f-865d-c582a62855d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1289 microversion: 2.1 time: 0.186821 May 19 12:05:58.445670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 30/60] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1289 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:58.454730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:58.455340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:58.471415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.475610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.476470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.480869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:58.481322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.482703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:58.510819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.511323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.512135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:58.512568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:58.871437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.894708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.902552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.906657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9ab66c29-7455-45ab-96f8-b81f128ec6e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1334 microversion: 2.20 time: 0.442736 May 19 12:05:58.907962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:58.910751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 31/61] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1334 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:05:58.926256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9aa1fec2-830a-4148-9a8b-01ad2f024879 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.475035 May 19 12:05:58.927371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 31/62] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:58 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:59.367827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:59.367827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:59.380596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.381624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.386172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.390528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.464414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:59.465221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:59.477372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.477818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.478662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.479087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.783398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:59.795202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b6be3df6-ce47-4c5a-b3ac-15b0e1389662 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1307 microversion: 2.1 time: 0.335616 May 19 12:05:59.795964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 32/63] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1307 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:59.930344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:59.931293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:59.936335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:05:59.943488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.944587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.945529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.946239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.947161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01bd9cae-602c-4b0e-9f97-a0f6b7479ae4 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.592065 May 19 12:05:59.948523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 32/64] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:59 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:05:59.956282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:05:59.957408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:05:59.964817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:05:59.967945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.968830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:05:59.970837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:05:59.972385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:00.056631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:00.091260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:00.094696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:00.097497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3aeb7b34-0f2c-4edb-84a6-7b7447bbb398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1340 microversion: 2.20 time: 0.175177 May 19 12:06:00.099785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 33/65] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:05:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1340 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:00.523401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:00.537755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7815925c-085d-416f-bc25-f2d173fe81a4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.587577 May 19 12:06:00.541865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 33/66] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:05:59 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:00.812678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:00.813181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:00.822532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:00.824567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:00.825775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:00.827807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:00.963385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:00.964087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:00.974541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:00.979930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:00.986400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:00.987014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.006619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:01.040291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4d0f9f1a-1084-45ab-bdaf-166a8197463d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1295 microversion: 2.1 time: 0.228016 May 19 12:06:01.043579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 34/67] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1295 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:01.119170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:01.120928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:01.130969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:01.133064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.135428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:01.137114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.268922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:01.280131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:01.285577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:01.287793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1ae3c430-0f4d-4506-a117-f9c058a02406 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1340 microversion: 2.20 time: 0.172987 May 19 12:06:01.303232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 35/68] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1340 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:01.476858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:01.489183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-85728ca5-c2ab-42fc-8ee4-7495b286d922 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.529555 May 19 12:06:01.490091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 34/69] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:00 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:01.560746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:01.562051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:01.578816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:01.579268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.580084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:01.580528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.736890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-b830a9ad-6fbd-44b4-b438-2e4fb273ba33 req-37597b92-7588-4f21-b2a1-0dcfce5b5631 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7fafb49c-cc35-4a08-b5e5-70871f75cc68", "name": "network-vif-plugged", "status": "completed", "tag": "3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:01.750396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b830a9ad-6fbd-44b4-b438-2e4fb273ba33 req-37597b92-7588-4f21-b2a1-0dcfce5b5631 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:01.751235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b830a9ad-6fbd-44b4-b438-2e4fb273ba33 req-37597b92-7588-4f21-b2a1-0dcfce5b5631 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:01.766975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-b830a9ad-6fbd-44b4-b438-2e4fb273ba33 req-37597b92-7588-4f21-b2a1-0dcfce5b5631 service nova] Creating event network-vif-plugged:3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4 for instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 on hv-cinder-80403 May 19 12:06:01.777204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-b830a9ad-6fbd-44b4-b438-2e4fb273ba33 req-37597b92-7588-4f21-b2a1-0dcfce5b5631 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.472203 May 19 12:06:01.778472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 36/70] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:02.023108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:02.032229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b8831a5a-199c-46a7-a7d3-dcb18395cf1b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.475350 May 19 12:06:02.034956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 35/71] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:01 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:02.062331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:02.063188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:02.100591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.102813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.105933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.110442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.310906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:02.313747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:02.339215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.341104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.344186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.345153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.493366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:02.507704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2081e1ec-9ab2-4591-b414-85cdc97490de tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.450392 May 19 12:06:02.509108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 37/72] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:02.518757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:02.519373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:02.526446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.527182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.528274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:02.528551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:02.777369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:02.789086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3435ef4-d1b4-4756-8130-11e4a2675369 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.278595 May 19 12:06:02.792244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 38/73] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:02 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:03.050898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:03.051500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:03.065844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.066513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.067033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.067511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.283894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:03.296774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-26617e2f-e316-464a-88e4-5c449d9559f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.249618 May 19 12:06:03.298403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 39/74] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:03 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:03.526409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:03.527355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:03.538238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.538764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.539533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.541535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.547307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:03.559445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:03.561475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:03.562848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-69fedf1a-c60f-4397-90df-ac77b2436db5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 1.258697 May 19 12:06:03.564166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 36/75] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 1260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:03.808411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:03.809196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:03.818143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:03.820063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.824387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.826773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.831587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:03.908095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=7 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:03.947028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:03.960631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-df030297-1bbd-4b58-b141-1d85c6dbc359 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.438523 May 19 12:06:03.971347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 40/76] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:03.976524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-cb7995e9-e58e-4327-b1ab-a31b7021b89f req-c5e733c9-6410-4089-a75f-60b8fbff95ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da2ea452-9604-40b7-b9cb-65be5fa21be5", "name": "network-vif-plugged", "status": "completed", "tag": "c5bf0325-0a37-4c7a-a88a-f713bf437054"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:03.986985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-cb7995e9-e58e-4327-b1ab-a31b7021b89f req-c5e733c9-6410-4089-a75f-60b8fbff95ee service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:03.987859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-cb7995e9-e58e-4327-b1ab-a31b7021b89f req-c5e733c9-6410-4089-a75f-60b8fbff95ee service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.004100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-cb7995e9-e58e-4327-b1ab-a31b7021b89f req-c5e733c9-6410-4089-a75f-60b8fbff95ee service nova] Creating event network-vif-plugged:c5bf0325-0a37-4c7a-a88a-f713bf437054 for instance da2ea452-9604-40b7-b9cb-65be5fa21be5 on hv-cinder-80403 May 19 12:06:04.009760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-cb7995e9-e58e-4327-b1ab-a31b7021b89f req-c5e733c9-6410-4089-a75f-60b8fbff95ee service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037097 May 19 12:06:04.011218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 41/77] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:04.279950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-27bc2d66-03ad-44a5-9fe0-0ddb0ad198a1 req-5e325833-33ea-417b-a2c1-e734746c569d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4518b411-8fac-48f2-8826-7b91651826b1", "tag": "cf5a95fb-957c-4a6a-8f4e-61dcbbae636e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:04.315230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:04.315536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-27bc2d66-03ad-44a5-9fe0-0ddb0ad198a1 req-5e325833-33ea-417b-a2c1-e734746c569d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.316794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-27bc2d66-03ad-44a5-9fe0-0ddb0ad198a1 req-5e325833-33ea-417b-a2c1-e734746c569d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.324765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e86a5577-281a-4408-998f-98320dcd9794 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1315 microversion: 2.1 time: 0.520378 May 19 12:06:04.327722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 37/78] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:03 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1315 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:04.351319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:04.352182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:04.366286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-27bc2d66-03ad-44a5-9fe0-0ddb0ad198a1 req-5e325833-33ea-417b-a2c1-e734746c569d service nova] Creating event network-changed:cf5a95fb-957c-4a6a-8f4e-61dcbbae636e for instance 4518b411-8fac-48f2-8826-7b91651826b1 on hv-cinder-80403 May 19 12:06:04.376270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.376748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.377526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.381323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.388491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-27bc2d66-03ad-44a5-9fe0-0ddb0ad198a1 req-5e325833-33ea-417b-a2c1-e734746c569d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.105507 May 19 12:06:04.390081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 42/79] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:04.582032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:04.582770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:04.614190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.619378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.621700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.625186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.664376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:04.684433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae77e201-8b45-461e-9491-429bc2cf57b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.354868 May 19 12:06:04.689288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 38/80] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:04 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:04.930864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:04.941171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:04.947768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:04.948970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ba72859-d0f5-424c-b9a5-507ceaf88a23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.371408 May 19 12:06:04.949533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 43/81] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:04.980802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:04.981861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:04.993175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.994214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:04.995056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:04.995476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.219079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:05.228026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-17192ba9-d5ea-4852-9093-633ad6667ca7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.251964 May 19 12:06:05.237677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 39/82] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:05.345890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:05.346514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:05.352564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.353187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.353963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.354350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.512016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:05.523919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c9c9cee-50f4-446b-b2a4-2b60df09b1c3 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1503 microversion: 2.1 time: 0.181788 May 19 12:06:05.524847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 44/83] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:05 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1503 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:05.705575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:05.706222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:05.714080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.714838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.716027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.716490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.974120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:05.975279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:05.990372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.991451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:05.993225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:05.994740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:06.082785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:06.103938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3006f92b-3b3f-4320-ac1f-233dcc349e18 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.402915 May 19 12:06:06.106629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 40/84] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:05 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:06.247032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:06.247678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:06.259286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:06.260896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:06.262496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:06.264770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:06.277236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:06.288925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:06.289850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:06.292088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:06.293548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ed8bf515-5258-41ed-bf60-2439658d7f4a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.330352 May 19 12:06:06.294273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 45/85] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:06.397909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-7976676b-43ef-420a-95c1-34ab086a45fd req-2884feb9-b51e-4a22-8201-fd6ec7f5f8ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a9117bef-13c8-47cc-bb42-ea7a47d5025a", "name": "network-vif-plugged", "status": "completed", "tag": "fc1f75e7-0085-4916-9009-12d6f71197eb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:06.413755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-7976676b-43ef-420a-95c1-34ab086a45fd req-2884feb9-b51e-4a22-8201-fd6ec7f5f8ea service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:06.414179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-7976676b-43ef-420a-95c1-34ab086a45fd req-2884feb9-b51e-4a22-8201-fd6ec7f5f8ea service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:06.427952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-7976676b-43ef-420a-95c1-34ab086a45fd req-2884feb9-b51e-4a22-8201-fd6ec7f5f8ea service nova] Creating event network-vif-plugged:fc1f75e7-0085-4916-9009-12d6f71197eb for instance a9117bef-13c8-47cc-bb42-ea7a47d5025a on hv-cinder-80403 May 19 12:06:06.434961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-7976676b-43ef-420a-95c1-34ab086a45fd req-2884feb9-b51e-4a22-8201-fd6ec7f5f8ea service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042157 May 19 12:06:06.435573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 46/86] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:06:06.472508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-2a1c1519-d786-4e02-b704-717c8eed14d7 req-afa61664-aa3b-41ba-ba9b-d6744c237afe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:06.483031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2a1c1519-d786-4e02-b704-717c8eed14d7 req-afa61664-aa3b-41ba-ba9b-d6744c237afe service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:06.483462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2a1c1519-d786-4e02-b704-717c8eed14d7 req-afa61664-aa3b-41ba-ba9b-d6744c237afe service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:06.496344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-2a1c1519-d786-4e02-b704-717c8eed14d7 req-afa61664-aa3b-41ba-ba9b-d6744c237afe service nova] Creating event network-changed:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:06:06.504435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-2a1c1519-d786-4e02-b704-717c8eed14d7 req-afa61664-aa3b-41ba-ba9b-d6744c237afe service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038589 May 19 12:06:06.505471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 47/87] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:06.516382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:06.527336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-264ac284-0285-4685-a318-b4532266cd5f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.285511 May 19 12:06:06.528103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 41/88] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:07.102673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b43fde62-2bcc-4c84-bccf-06220bfaf898"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:07.103811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:07.109971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.110412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.111158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.111580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.119026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:07.119632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:07.125815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.126269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.127073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.127452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.276909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:07.285960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9e7adce2-bfd6-4b34-97c7-e5ae35b7dc24 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.170471 May 19 12:06:07.286694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 42/89] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:07 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:07.311378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:07.311968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:07.318141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.318574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.319316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.319734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.502011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:07.513374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:07.515217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:07.516333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34ea7b4a-028b-4774-99a2-0de97add9ed9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.208993 May 19 12:06:07.517449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 43/90] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:07.544057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:07.544680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:07.550872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.551250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.552008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:07.552430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:07.685896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:07.694249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-95531d5e-46ad-4754-bdea-4015eb84d068 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.153946 May 19 12:06:07.695253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 44/91] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:08.304110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:08.305413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:08.312357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.312834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.313589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.314076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.449656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:08.458035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6a1e13d8-0e03-4b7b-9362-224e61fede20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.158542 May 19 12:06:08.458821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 45/92] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:08 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:08.534554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:08.535268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:08.541650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.542092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.542873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.543314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.678238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:08.686280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:08.687742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:08.688872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c64020ba-79a4-4f42-8f7a-1c74a54802cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.158274 May 19 12:06:08.689415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 46/93] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:08.711254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:08.711847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:08.717797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.718231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.718970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:08.719469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:08.850893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:08.858695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-87371bfd-24dd-4822-a682-3da7e2368931 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.151147 May 19 12:06:08.859360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 47/94] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:09.475338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:09.476027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:09.482318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.482760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:09.483510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.483938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:09.625920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:09.633963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-140787bf-9721-49bd-8e5b-2fa629d39a00 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.162776 May 19 12:06:09.634542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 48/95] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:09 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:09.707312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:09.707999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:09.714797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.715306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:09.716262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.716796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:09.852631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:09.860949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:09.862446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:09.863556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-932bcdc6-9dee-4520-856f-a9e8c6a7a300 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.160616 May 19 12:06:09.864128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 49/96] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:09.878739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:09.879340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:09.886177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.886177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:09.886703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:09.887139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:10.070648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:10.080303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-89609bf1-394e-4bfc-893e-1fa817d7ae03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.205044 May 19 12:06:10.081245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 50/97] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:10.658277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:10.658277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:10.659618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:10.669068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:10.670197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:10.671019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:10.818517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:10.826103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c1ad3997-baad-44d3-be6b-6541d1fe4ff1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.178928 May 19 12:06:10.826680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 51/98] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:10 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:10.884527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:10.885171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:10.891729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:10.892137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:10.892956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:10.893407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:11.028990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:11.037102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:11.038563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:11.039675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-11f84cdb-6676-43ef-ae35-8f5c670dccc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.159033 May 19 12:06:11.040243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 52/99] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:11.097411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:11.098012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:11.104078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:11.104527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:11.105295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:11.105723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:11.246498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:11.254559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-287aa76f-c981-4efd-9abd-0e6abce2999e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.161035 May 19 12:06:11.255134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 53/100] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:11.843131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:11.843811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:11.850895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:11.851545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:11.852314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:11.852776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:11.988427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:11.995978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3eb882a7-6759-4ea2-bb71-c3ba6e98e287 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.156847 May 19 12:06:11.996549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 54/101] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:11 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:12.061221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:12.062250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:12.072392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:12.073182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:12.074421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:12.075130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:12.212954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:12.223187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:12.225350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:12.227032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-02057c6b-ffda-4d51-bd43-0488de408588 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.171861 May 19 12:06:12.227821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 55/102] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:12.271265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:12.271854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:12.278167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:12.278608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:12.279350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:12.279769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:12.415099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:12.426820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9807283d-b338-4c15-97ee-633d4d0b1a85 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.159066 May 19 12:06:12.427580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 56/103] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:13.013006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:13.014137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:13.020187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.020734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.021545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.022058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.162127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:13.169903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d7a676af-f4e8-46ee-a4b8-c3a9c8e6163f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1321 microversion: 2.1 time: 0.160894 May 19 12:06:13.170505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 57/104] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:13 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1321 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:13.249614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:13.249614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:13.253770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.254390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.255567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.256172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.262743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:13.376724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9a216a6d-3038-4861-bb7d-2bb996064b98 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "POST /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 6.277578 May 19 12:06:13.378376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 48/105] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:06:07 2022] POST /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68/os-volume_attachments => generated 194 bytes in 6280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:13.446360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:13.447293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:13.454934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.455594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.456446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:13.456961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:13.659319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:13.670055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:13.675980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e54fb7e6-4982-41eb-ba1b-a4925cd810b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.235685 May 19 12:06:13.678034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 49/106] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:13.679345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:13.680845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:13.682283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3f4f400c-8870-419a-9334-d87db8f5001d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.441053 May 19 12:06:13.682972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 58/107] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:14.191339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:14.192407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:14.202722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.203167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.203926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.204372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.346414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:14.354653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7e6fee93-95c2-44c6-9e56-f807b0832a9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1516 microversion: 2.1 time: 0.167773 May 19 12:06:14.355155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 50/108] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:14 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1516 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:14.372057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:14.373912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:14.380921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.381570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.382408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.382863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.446740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'flavor' on Instance uuid a9117bef-13c8-47cc-bb42-ea7a47d5025a {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:06:14.694731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:14.695776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:14.702452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.704363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.704363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.704363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.717728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cafdf682-879d-43b1-a743-adc8508f2d8c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a/action" status: 200 len: 32 microversion: 2.1 time: 0.349904 May 19 12:06:14.718334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 59/109] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:06:14 2022] POST /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a/action => generated 32 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:06:14.723370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:14.724046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:14.730171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.730595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.731361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.731780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.839730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:14.848844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c03f4c3f-b7f7-47d2-88d5-49b9ceb8959e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.158098 May 19 12:06:14.858287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 51/110] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:14.858287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:14.858287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:14.861380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.861380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.864915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:14.864915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:14.870458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:14.876377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:14.885658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:14.887131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:14.888237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-59119931-d8e5-4475-bdc2-05d4c9109d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.168860 May 19 12:06:14.889329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 60/111] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:14.997391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:15.005526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1b6eb7f0-618d-42b7-89da-1625f8f8b842 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.155126 May 19 12:06:15.006078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 52/112] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:14 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:15.867483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:15.868089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:15.875286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:15.875709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:15.876469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:15.876923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:15.905967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:15.906530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:15.913695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:15.914145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:15.914911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:15.915338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:16.012630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:16.021201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f7336fed-7bc3-4741-9159-b848b246c9ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.158226 May 19 12:06:16.021883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 61/113] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:16.026409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:16.026998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:16.032997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:16.033435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:16.034182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:16.034604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:16.053667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:16.061775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:16.063373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:16.064516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a26b678e-86ae-4b01-89f7-9d00f17c3b0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.162464 May 19 12:06:16.065114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 53/114] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:16.166261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:16.173869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1fa8e2f9-e8d5-4d35-9ea9-d01d50aa907e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.151031 May 19 12:06:16.174455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 62/115] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:16 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:17.038020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:17.038624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:17.046108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.046558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.047305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.047729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.083369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:17.090173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:17.103711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.104175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.104990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.105421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.283278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:17.294812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:17.296346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:17.297537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-427a1567-6510-4ae6-a185-160c77f5c932 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.218847 May 19 12:06:17.301421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 63/116] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:17.306743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:17.307847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:17.314814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.315254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.316015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:17.316470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:17.358332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:17.366160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0b4b15eb-ee81-4978-91ef-0894dabc932f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.332058 May 19 12:06:17.366910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 54/117] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:17.448255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:17.456017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-23d55329-4248-480f-96bb-ec666b0c26d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.153418 May 19 12:06:17.456612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 64/118] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:17 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:18.320845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:18.322915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:18.331072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.331550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.332285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.332887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.386137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:18.388948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:18.396163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.396958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.398869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.399556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.569066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:18.586438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:18.589806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:18.592246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-37ac1c23-b668-4bcb-ba4d-6c11b9995926 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.276843 May 19 12:06:18.595570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 55/119] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:18.609947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:18.611877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:18.629374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.629720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.630510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.630943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.730036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:18.742509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b8a963e3-5c8a-4477-8405-7a0f4bce22b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.362352 May 19 12:06:18.743535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 65/120] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:18.859371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-9a4f0a5f-24be-485f-a916-3632c51d6691 req-ffa6741e-777d-4be4-b6c7-3fb36a89962c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "name": "network-vif-plugged", "status": "completed", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:18.868108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:18.871015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-9a4f0a5f-24be-485f-a916-3632c51d6691 req-ffa6741e-777d-4be4-b6c7-3fb36a89962c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:18.872341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-9a4f0a5f-24be-485f-a916-3632c51d6691 req-ffa6741e-777d-4be4-b6c7-3fb36a89962c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:18.890374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-9a4f0a5f-24be-485f-a916-3632c51d6691 req-ffa6741e-777d-4be4-b6c7-3fb36a89962c service nova] Creating event network-vif-plugged:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:06:18.897062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-9a4f0a5f-24be-485f-a916-3632c51d6691 req-ffa6741e-777d-4be4-b6c7-3fb36a89962c service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042046 May 19 12:06:18.898225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 66/121] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:06:18.941399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:18.949996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a68aa4a3-4d3d-45ee-838f-b2be3cb9305a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.353069 May 19 12:06:18.950590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 56/122] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:18 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:19.614101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:19.615769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:19.623277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.623711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:19.624707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.625216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:19.759535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:19.760411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:19.762881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=80 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:19.768877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.769320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:19.770131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.770769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:19.808366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:19.818340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:19.819833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:19.820986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ef6067b8-c345-4843-af4c-40581515a45f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.212262 May 19 12:06:19.821735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 67/123] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:19.967623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:19.967836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:19.974324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.974757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:19.975509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:19.975927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:20.122685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:20.130724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f29b230d-0c39-42eb-8fe1-df574e7594f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.167459 May 19 12:06:20.131304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 68/124] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:19 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:20.185265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:20.193089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-df9516a8-4656-4be8-8eb7-a9559a19a2cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.437468 May 19 12:06:20.193635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 57/125] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:20.839526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:20.840127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:20.851229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:20.851675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:20.852451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:20.852908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:21.017692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:21.025706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:21.027196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:21.028305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-631b1718-9b93-424e-822f-e6569f0c8345 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.192651 May 19 12:06:21.029064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 69/126] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:21.147226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:21.147993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:21.156153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:21.156873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:21.157636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:21.158056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:21.210049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:21.211179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:21.218159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:21.218828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:21.219857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:21.220523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:21.339955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:21.347722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b47f2fbc-5bec-4039-999d-3979ba11ed91 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.204410 May 19 12:06:21.348366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 58/127] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:21 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:21.402932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:21.413910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d4cdc2af-59dd-4e0b-9bcd-f95cd6807b29 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.207690 May 19 12:06:21.414481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 70/128] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:22.046360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:22.046958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:22.053333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.053775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.054528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.055112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.193148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:22.202171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:22.203633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:22.204825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8f199fdd-2c95-433a-beb1-03061d4f23a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.162347 May 19 12:06:22.205391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 59/129] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:22.365224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:22.365938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:22.383847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.384886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.385815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.386239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.432553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:22.441268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:22.447658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.448178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.449106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:22.449731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:22.553374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:22.562289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42c696ab-522e-4158-b3f6-56f62c7c472a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.201346 May 19 12:06:22.565834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 71/130] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:22 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:22.674470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:22.682275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f60edcb6-94d7-4e9d-b029-55b64d5a111e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.254946 May 19 12:06:22.697682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 60/131] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:23.229726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:23.231012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:23.245507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.246061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.246828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.247254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.418016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:23.427943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:23.429690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:23.430812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6cef8a28-fb7a-46ff-aa05-d67228c3ceef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1408 microversion: 2.20 time: 0.210975 May 19 12:06:23.431370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 72/132] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1408 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:06:23.582750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:23.583509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:23.590461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.590903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.591945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.592812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.703351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:23.703471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:23.711485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.712382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.713575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:23.714293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:23.742890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:23.743099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:23.753726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e6c88558-d43f-4fde-86d2-15beacc7e740 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.175174 May 19 12:06:23.755362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 61/133] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:23 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:23.913080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:23.921175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5285112d-aaef-49d9-9b77-08876a08620d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.225733 May 19 12:06:23.921936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 73/134] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:24.449110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:24.449790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:24.457291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.457922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.458678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.459250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.602058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:24.615117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:24.617040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:24.618326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4d40e59e-a388-45e1-8301-e529b7de5d46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1598 microversion: 2.20 time: 0.173045 May 19 12:06:24.618889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 62/135] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:06:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1598 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:06:24.636825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:24.637373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:24.644251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.644750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.645527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.645935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.774538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab638536-259b-47ba-ad2a-3ff672d52955 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.141299 May 19 12:06:24.781501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 74/136] 10.222.0.35 () {60 vars in 1315 bytes} [Thu May 19 12:06:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-interface => generated 284 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:06:24.781853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:24.781853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:24.788704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.789576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.790235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.791244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.874818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-3b0cd0b2-59db-46bf-835d-999df4692614 req-4b158152-d418-4edb-a676-8bf5fda1a4e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4518b411-8fac-48f2-8826-7b91651826b1", "name": "network-vif-plugged", "status": "completed", "tag": "cf5a95fb-957c-4a6a-8f4e-61dcbbae636e"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:24.892764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-3b0cd0b2-59db-46bf-835d-999df4692614 req-4b158152-d418-4edb-a676-8bf5fda1a4e3 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.893209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-3b0cd0b2-59db-46bf-835d-999df4692614 req-4b158152-d418-4edb-a676-8bf5fda1a4e3 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.910335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-3b0cd0b2-59db-46bf-835d-999df4692614 req-4b158152-d418-4edb-a676-8bf5fda1a4e3 service nova] Creating event network-vif-plugged:cf5a95fb-957c-4a6a-8f4e-61dcbbae636e for instance 4518b411-8fac-48f2-8826-7b91651826b1 on hv-cinder-80403 May 19 12:06:24.929211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-3b0cd0b2-59db-46bf-835d-999df4692614 req-4b158152-d418-4edb-a676-8bf5fda1a4e3 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057823 May 19 12:06:24.929829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 75/137] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:06:24.937870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:24.938565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:24.959796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.959796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:24.960096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:24.960631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:25.252144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:25.265028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f7c5f369-f727-4680-ac7e-b37c8346bf9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.494673 May 19 12:06:25.275462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 63/138] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:24 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:25.369440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:25.380142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fbf6748a-da2e-4742-a431-fbb30c234b9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.446236 May 19 12:06:25.383717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 76/139] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:26.283234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:26.284722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:26.293340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:26.294167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:26.295155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:26.295821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:26.403166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:26.403800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:26.415207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:26.415654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:26.418476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:26.421319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:26.605701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:26.620370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e87cb032-c356-4140-a395-42698540de7e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.341158 May 19 12:06:26.622264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 64/140] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:26 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:26.681837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:26.690586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eca889ff-e309-4ff3-9938-96bb5b29b763 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.291853 May 19 12:06:26.693293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 77/141] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:27.261366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-f7f13c68-2c2f-4fae-a408-0b3c46504547 req-5494f922-ad23-423d-ad08-fec7bf7ce831 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:27.273493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-f7f13c68-2c2f-4fae-a408-0b3c46504547 req-5494f922-ad23-423d-ad08-fec7bf7ce831 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.274374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-f7f13c68-2c2f-4fae-a408-0b3c46504547 req-5494f922-ad23-423d-ad08-fec7bf7ce831 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.290742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-f7f13c68-2c2f-4fae-a408-0b3c46504547 req-5494f922-ad23-423d-ad08-fec7bf7ce831 service nova] Creating event network-changed:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:06:27.301056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-f7f13c68-2c2f-4fae-a408-0b3c46504547 req-5494f922-ad23-423d-ad08-fec7bf7ce831 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044327 May 19 12:06:27.303251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 65/142] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:27.428287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:27.437749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:27.461924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.462639 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.463438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.464261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.512188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:06:27.638662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:27.639714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:27.652312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.653060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.654822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.657135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.674096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-54d8ec54-dee6-4d8a-8343-013dd46e58d8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 204 len: 0 microversion: 2.1 time: 0.252190 May 19 12:06:27.676261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 78/143] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:06:27 2022] DELETE /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:06:27.710986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:27.711724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:27.725270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.730181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.731003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.731429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.934305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:27.955614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-40a183b2-35ea-49b6-b7af-d8d82e6b988d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.321572 May 19 12:06:27.956463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 66/144] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:27 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:27.976908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:27.978150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:27.988491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.989255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.990186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:27.990750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:27.997260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:28.011592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a965231b-1d27-46b3-894d-99d69afda0f8 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.311177 May 19 12:06:28.013677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 79/145] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:27 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:28.217260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:28.235200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-39af6709-529e-424f-b024-86a76735adf9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.274138 May 19 12:06:28.246388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 67/146] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:28.975242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:28.976889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:28.985342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:28.985904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:28.987073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:28.987875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:29.030632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:29.031492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:29.041336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:29.042367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:29.043709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:29.044389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:29.201386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:29.210165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fec420bd-b56a-41d5-bb7c-62f65faff747 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.184018 May 19 12:06:29.212263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 68/147] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:29 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:29.263347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:29.264193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:29.278592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:29.279053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:29.280630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:29.282861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:29.313398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:29.360094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:29.369405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-29c987c0-e327-4d94-9937-855e7f22e5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.399392 May 19 12:06:29.370585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 80/148] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:28 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:29.463077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:29.470909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-25984ffe-00ea-43e5-a75a-6a49f17a6464 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.217113 May 19 12:06:29.471485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 69/149] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:30.229178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:30.230017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:30.237073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.237628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.238653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.239267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.395956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:30.397482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:30.408895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:30.415062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.415701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.416848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.417422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.419147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-77a70308-e7bd-4f88-a878-41c653ff5ac6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.194211 May 19 12:06:30.433536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 81/150] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:30 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:30.488982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:30.489924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:30.497492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.498169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.499144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:30.499904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:30.631971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:30.641187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f29b116f-dca0-484f-947e-0fc52b0def28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.254153 May 19 12:06:30.642431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 70/151] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:30 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:30.735256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:30.743204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b41f0d68-8813-48e0-93ab-a13a795fa023 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.258517 May 19 12:06:30.753020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 82/152] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:31.439157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:31.439966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:31.447638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.448163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.449093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.449546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.610276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:31.619191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c8718dc-09be-4f8d-b8b4-fa5c35019df6 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.184850 May 19 12:06:31.620092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 71/153] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:31 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:31.658717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:31.659318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:31.667110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.667556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.668341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.668780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.760049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:31.760813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:31.775741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.777426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.780450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:31.780942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:31.816391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:31.835680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-49c5d5b3-b016-4b7f-b115-b5dd00ae2bc7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.180810 May 19 12:06:31.836295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 83/154] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:31 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:31.934267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:31.942524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4eb4f8cb-a38e-4e97-9123-09b096a1666e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.186837 May 19 12:06:31.944204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 72/155] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:32.636839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:32.637708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:32.645139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.645837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:32.646897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.647125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:32.833762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:32.847870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-be6f1922-c794-4ddf-95c7-2c12b3debdef tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.212594 May 19 12:06:32.848577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 84/156] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:32 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:32.853995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:32.854603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:32.861643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.862315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:32.863294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.864037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:32.959537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:32.963539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:32.973758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.979867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:32.985731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:32.986376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:33.045181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:33.053913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0cabb4b-153f-4924-8f4c-0e769887042f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.203748 May 19 12:06:33.056922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 73/157] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:32 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:33.133886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:33.141886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c9bc71ef-b171-4982-8b1e-b121fb714629 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1363 microversion: 2.1 time: 0.186267 May 19 12:06:33.142461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 85/158] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1363 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:33.864038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:33.864666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:33.871654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:33.872092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:33.872881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:33.873365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.010542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:34.019039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d24682ef-dcc9-4957-81f1-81301247a5ee tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.159277 May 19 12:06:34.019708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 74/159] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:33 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:34.071684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:34.072295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:34.078947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.079520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.080282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.080719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.161755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:34.162606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:34.171750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.172793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.173929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.174496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.237056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:34.248395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e0ff1cf-43e3-45c4-9ed7-e854615a4c0d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.180500 May 19 12:06:34.249462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 86/160] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:34 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:34.327645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:34.335345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63dafb72-0d23-43f1-9d58-0eb9637b302a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1548 microversion: 2.1 time: 0.179591 May 19 12:06:34.335926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 75/161] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1548 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:34.353379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:34.353857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:34.359855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.360295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.361058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:34.361474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:34.439130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-496a26cd-1084-431a-9aa4-9248203db6c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089817 May 19 12:06:34.439732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 87/162] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:06:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:35.037392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:35.038098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:35.044512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:35.045092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:35.045931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:35.046372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:35.187941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:35.196115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-75a54694-e3b7-4e90-bd58-db1512171e01 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.163288 May 19 12:06:35.196705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 76/163] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:35 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:35.265924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:35.266652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:35.273239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:35.273626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:35.274380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:35.274809 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:35.673315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:35.681162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-90d5e46b-b996-4189-aa7e-24c6959b7ece tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.419405 May 19 12:06:35.681719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 88/164] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:35 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:35.836987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-7e8d870e-0f18-49ed-a63f-b5b126a2359e req-021408ca-6b6e-4841-872d-dcf0718f40b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4518b411-8fac-48f2-8826-7b91651826b1", "tag": "cf5a95fb-957c-4a6a-8f4e-61dcbbae636e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:35.845218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-7e8d870e-0f18-49ed-a63f-b5b126a2359e req-021408ca-6b6e-4841-872d-dcf0718f40b8 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:35.845719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-7e8d870e-0f18-49ed-a63f-b5b126a2359e req-021408ca-6b6e-4841-872d-dcf0718f40b8 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:35.856687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-7e8d870e-0f18-49ed-a63f-b5b126a2359e req-021408ca-6b6e-4841-872d-dcf0718f40b8 service nova] Creating event network-changed:cf5a95fb-957c-4a6a-8f4e-61dcbbae636e for instance 4518b411-8fac-48f2-8826-7b91651826b1 on hv-cinder-80403 May 19 12:06:35.862011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-7e8d870e-0f18-49ed-a63f-b5b126a2359e req-021408ca-6b6e-4841-872d-dcf0718f40b8 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028782 May 19 12:06:35.862701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 77/165] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:36.213558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:36.214432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:36.220533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:36.221009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:36.221764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:36.222222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:36.361371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:36.363692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:36.373681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-76d2bd77-9cdd-4dd0-b2ed-bd792e5fffdd tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.163736 May 19 12:06:36.374432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 89/166] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:36 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:36.705060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:36.706907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:36.723055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:36.724694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:36.726410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:36.728490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:36.744055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-3932891b-1397-4309-bf11-ad0668c2aabe req-b5ce9398-96fd-4dcb-a757-f35dd5f285a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7fafb49c-cc35-4a08-b5e5-70871f75cc68", "name": "network-vif-unplugged", "status": "completed", "tag": "3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:36.753730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-3932891b-1397-4309-bf11-ad0668c2aabe req-b5ce9398-96fd-4dcb-a757-f35dd5f285a0 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:36.754367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-3932891b-1397-4309-bf11-ad0668c2aabe req-b5ce9398-96fd-4dcb-a757-f35dd5f285a0 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:36.768565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-3932891b-1397-4309-bf11-ad0668c2aabe req-b5ce9398-96fd-4dcb-a757-f35dd5f285a0 service nova] Creating event network-vif-unplugged:3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4 for instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 on hv-cinder-80403 May 19 12:06:36.775303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-3932891b-1397-4309-bf11-ad0668c2aabe req-b5ce9398-96fd-4dcb-a757-f35dd5f285a0 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035725 May 19 12:06:36.776438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 90/167] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:36.894523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:36.905176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c009c2f-8021-4b05-9c57-38ae15d47088 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.206192 May 19 12:06:36.906653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 78/168] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:36 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:37.398242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:37.401625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:37.411120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:37.412191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:37.413545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:37.414346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:37.734948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:37.748934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4359d128-c79a-4ef7-8f6e-9dd3d5033e37 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1555 microversion: 2.1 time: 0.358939 May 19 12:06:37.750821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 91/169] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:37 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1555 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:37.922660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:37.923255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:37.929840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:37.930354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:37.931015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:37.931430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:37.957872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-7137de49-ce3b-4401-8a4d-b3c480aac171 req-8f365897-7eaa-40e2-9c3a-f8bbe5c7819c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7fafb49c-cc35-4a08-b5e5-70871f75cc68", "name": "network-vif-deleted", "tag": "3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:37.987340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-7137de49-ce3b-4401-8a4d-b3c480aac171 req-8f365897-7eaa-40e2-9c3a-f8bbe5c7819c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:37.987811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-7137de49-ce3b-4401-8a4d-b3c480aac171 req-8f365897-7eaa-40e2-9c3a-f8bbe5c7819c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:38.006819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-7137de49-ce3b-4401-8a4d-b3c480aac171 req-8f365897-7eaa-40e2-9c3a-f8bbe5c7819c service nova] Creating event network-vif-deleted:3c0aee1c-bdfc-4fb4-8288-88284eb8eaa4 for instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 on hv-cinder-80403 May 19 12:06:38.011591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-7137de49-ce3b-4401-8a4d-b3c480aac171 req-8f365897-7eaa-40e2-9c3a-f8bbe5c7819c service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063042 May 19 12:06:38.012195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 92/170] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:06:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:06:38.079255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:38.088624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0276103-65d8-4cf0-b71b-3f7472eb4cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.169669 May 19 12:06:38.089271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 79/171] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:37 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:38.771835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:38.772541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:38.799070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:38.800896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:38.803282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:38.804663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:38.885382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3fc0a97d39d04020b8f933dbd5a57933', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb8c6f8383ec43868ace8748bdd3946b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:38.894046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f3a862e8-4601-4bdb-b9f5-ae8944688f85 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 200 len: 1346 microversion: 2.1 time: 0.128480 May 19 12:06:38.894837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 93/172] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:38 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 1346 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:39.107985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:39.109409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:39.120632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:39.121416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:39.122279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:39.123464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:39.311510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:39.320838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dd0f4e42-00b7-4d67-b64d-5a0e139bd5bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.216872 May 19 12:06:39.322083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 80/173] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:39 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:39.915221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:39.918823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:39.925524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:39.926042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:39.926864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:39.927354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:39.943931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] HTTP exception thrown: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Request method failure captured: May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 HTTP/1.1 May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKCx1CDyb8HSyKDFckgpOsQEHesSqMZZ9-UOT0h2tleKM4W90n9E5C-5itrkHwRmmkJGyQDrXILubHh-cfkYLvmHx2BuhyJgDEAt7GtKXCfmWJmnSClX3n-uCAMApaOm3uWF0F0PNyQzQKxoZjNEAQf58QBbfe_XYW-Xp2snIN7gAQ May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:06:39.949028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}] May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 3fc0a97d39d04020b8f933dbd5a57933 May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:39.950449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '7fafb49c-cc35-4a08-b5e5-70871f75cc68'} May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:06:39.951284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:39.952592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:39.957889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Returning 404 to user: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:06:39.957889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-abb7e959-f7f8-4aac-ab4a-6767acc9b860 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 404 len: 111 microversion: 2.1 time: 0.044529 May 19 12:06:39.957889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 94/174] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:39 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:06:40.063876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:40.065389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:40.072352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:40.072962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:40.073959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:40.074384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:40.090223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] HTTP exception thrown: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Request method failure captured: May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68/os-volume_attachments HTTP/1.1 May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKCx1CDyb8HSyKDFckgpOsQEHesSqMZZ9-UOT0h2tleKM4W90n9E5C-5itrkHwRmmkJGyQDrXILubHh-cfkYLvmHx2BuhyJgDEAt7GtKXCfmWJmnSClX3n-uCAMApaOm3uWF0F0PNyQzQKxoZjNEAQf58QBbfe_XYW-Xp2snIN7gAQ May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:06:40.095367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}] May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 3fc0a97d39d04020b8f933dbd5a57933 May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:40.096612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '7fafb49c-cc35-4a08-b5e5-70871f75cc68'} May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:06:40.097531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:40.102173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:40.104287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Returning 404 to user: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:06:40.104287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4746bd8c-f2bc-4f0d-98d5-84bab0c3196d tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.037920 May 19 12:06:40.104287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 81/175] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:06:40 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68/os-volume_attachments => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:06:40.341632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:40.343276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:40.357259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:40.358421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:40.360047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:40.360996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:40.561712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:40.570304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6f0809cd-964b-4047-a0bc-1df004e117cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.235832 May 19 12:06:40.571288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 95/176] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:40 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:41.338360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:41.338941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:41.349779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.350833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:41.353342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.356943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:41.416669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] HTTP exception thrown: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Request method failure captured: May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 HTTP/1.1 May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKCx1CDyb8HSyKDFckgpOsQEHesSqMZZ9-UOT0h2tleKM4W90n9E5C-5itrkHwRmmkJGyQDrXILubHh-cfkYLvmHx2BuhyJgDEAt7GtKXCfmWJmnSClX3n-uCAMApaOm3uWF0F0PNyQzQKxoZjNEAQf58QBbfe_XYW-Xp2snIN7gAQ May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:06:41.418866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}] May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 3fc0a97d39d04020b8f933dbd5a57933 May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.420107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '7fafb49c-cc35-4a08-b5e5-70871f75cc68'} May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:06:41.420767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.421806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:06:41.422863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Returning 404 to user: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:06:41.423750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ed0c6a51-4742-4a3d-b6da-66f76501fe64 tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 404 len: 111 microversion: 2.1 time: 0.086852 May 19 12:06:41.424570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 82/177] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:06:41 2022] DELETE /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:06:41.433135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:41.433775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] [instance: 7fafb49c-cc35-4a08-b5e5-70871f75cc68] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:41.439943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.440384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:41.442684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.443692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:41.482073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] HTTP exception thrown: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Request method failure captured: May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 HTTP/1.1 May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKCx1CDyb8HSyKDFckgpOsQEHesSqMZZ9-UOT0h2tleKM4W90n9E5C-5itrkHwRmmkJGyQDrXILubHh-cfkYLvmHx2BuhyJgDEAt7GtKXCfmWJmnSClX3n-uCAMApaOm3uWF0F0PNyQzQKxoZjNEAQf58QBbfe_XYW-Xp2snIN7gAQ May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:06:41.484701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_bb8c6f8383ec43868ace8748bdd3946b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/bb8c6f8383ec43868ace8748bdd3946b"}]}] May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: bb8c6f8383ec43868ace8748bdd3946b May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-DeleteServersTestJSON-1205791923 May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 3fc0a97d39d04020b8f933dbd5a57933 May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-DeleteServersTestJSON-1205791923-project May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.486495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '7fafb49c-cc35-4a08-b5e5-70871f75cc68'} May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:06:41.487274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. May 19 12:06:41.488860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:06:41.492417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] Returning 404 to user: Instance 7fafb49c-cc35-4a08-b5e5-70871f75cc68 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:06:41.500513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3112244d-c320-4632-b377-6e5a2a60966e tempest-DeleteServersTestJSON-1205791923 tempest-DeleteServersTestJSON-1205791923-project] 10.222.0.35 "GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68" status: 404 len: 111 microversion: 2.1 time: 0.068058 May 19 12:06:41.503175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 96/178] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:41 2022] GET /compute/v2.1/servers/7fafb49c-cc35-4a08-b5e5-70871f75cc68 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:06:41.588915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:41.589906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:41.604984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.607824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:41.610611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:41.612393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:42.301849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:42.312626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e27ec39-5d00-4a23-aa6c-3a14648a3dd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.728723 May 19 12:06:42.317764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 83/179] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:41 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:43.335307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:43.335942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:43.346077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:43.349928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:43.350963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:43.352924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:43.626087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:43.636864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c53c6365-ed79-4037-8826-b51d6a4d4b14 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.306687 May 19 12:06:43.640026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 97/180] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:43 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:44.657492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:44.658323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:44.672698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:44.674398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:44.675720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:44.677294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:45.098117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:45.112522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-33647112-00c2-4296-84a4-0aa1f471c872 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.459669 May 19 12:06:45.124143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 84/181] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:44 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:46.129749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:46.130500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:46.141128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:46.144350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:46.145565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:46.146019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:46.423387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:46.434219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-47a16e1d-7dde-40fd-a88a-13833ccf5376 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.308269 May 19 12:06:46.438634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 98/182] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:46 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:47.454393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:47.456863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:47.465222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:47.465656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:47.466414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:47.466835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:47.645540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:47.655850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63e871ce-5a6c-4046-9d90-4f1f22aaa805 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.207729 May 19 12:06:47.657002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 85/183] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:47 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:48.675092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:48.676495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:48.705451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:48.706618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:48.709826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:48.713125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:49.007004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:49.017194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-99dc9c47-1281-4748-8d6d-597f396a0f87 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.347662 May 19 12:06:49.018067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 99/184] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:48 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:50.036533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:50.037888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:50.044847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=58 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:50.063126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:50.063831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:50.067567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:50.068245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:50.473111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:50.481574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-564ae9a3-d996-4655-ad70-743414170aaf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.450250 May 19 12:06:50.491396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 86/185] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:50 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:51.509312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:51.509981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:51.520379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:51.523790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:51.523790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:51.525515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:51.526212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:51.747463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:51.755879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-028b65e8-cfd8-4684-b8e9-c3ff14b01970 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.260973 May 19 12:06:51.756877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 100/186] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:51 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:52.790763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:52.793831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:52.812810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:52.813742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:52.818992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:52.819698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:53.337560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:53.355923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3260f73b-70e2-40f0-94c9-5587f63421a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.585884 May 19 12:06:53.356879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 87/187] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:52 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:54.374432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:54.375222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:54.392465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:54.392980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:54.396482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:54.396823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:54.780988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:54.791756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8a488ccd-edb6-496a-83e6-fb78895f9b11 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.421208 May 19 12:06:54.792267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 101/188] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:54 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:55.206836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c84d1bb6-0e23-4b67-8bde-6cef487c3988 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Action: 'create', calling method: .version_select at 0x7fe20ccc68b0>, body: {"keypair": {"name": "tempest-TestShelveInstance-246156379", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBlvgfdnU6BOHD3yBokk7b5hrwQUJDMUxIDUUJq/6kZB94O1tcH7Q4GQIY8k/oXqHKtn0umfNPgU1eXydSbvjb1mYK/KOo3T+oI/OfngYLVrXCIDZXga+BKT0iNa+LMQrQ=="}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:55.248942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-c84d1bb6-0e23-4b67-8bde-6cef487c3988 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Getting quotas for project 1082ac24dda74201ab7a9bf670d83a38. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:06:55.253909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-c84d1bb6-0e23-4b67-8bde-6cef487c3988 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Getting quotas for user f8e5131aab4e43a18d98e719e83d6268 and project 1082ac24dda74201ab7a9bf670d83a38. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:06:55.309096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c84d1bb6-0e23-4b67-8bde-6cef487c3988 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.473735 May 19 12:06:55.310218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 88/189] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:06:54 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:06:55.809782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:55.810437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:55.818874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:55.820049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:55.823548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:55.825453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:56.082911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:56.091751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-77bdc0d1-0fca-49e8-948b-e85c005ccffb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.286296 May 19 12:06:56.096991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 102/190] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:55 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:57.108064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:57.108702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:57.132359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:57.132845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:57.133648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:57.134083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:57.152012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:57.538124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:57.554632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-80d5ecf9-f0c8-48fa-8480-2f2dc98ef45d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.448141 May 19 12:06:57.554632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 89/191] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:57 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:06:58.572539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:58.574051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:58.589797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:58.590255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:58.592452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:58.593844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:58.881524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:06:58.895188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9df4930c-6be3-42d0-a08f-de4176329b74 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.327827 May 19 12:06:58.897599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 103/192] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:58 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:06:59.648322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-91287976", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "networks": [{"uuid": "598be137-c8c2-4296-bb8a-3d04b858ef0b"}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:06:59.737756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] validate_networks() for [('598be137-c8c2-4296-bb8a-3d04b858ef0b', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:06:59.917621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:06:59.918253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:06:59.925634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:59.926085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:06:59.926890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:06:59.927325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:00.192840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:00.206569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b5ff7673-5839-43a5-8a96-61328ac18e1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.295218 May 19 12:07:00.207857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 104/193] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:06:59 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:00.873994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:07:00.874733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:07:00.875365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:07:00.876355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:07:00.877037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:07:00.895127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:00.895573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:00.896385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:00.896792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:00.945038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Getting quotas for project 92c49142ea324049ba24b55ff84e5a3b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:00.951011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Getting quotas for user d059ba56941148f3bb6080ba4f537f9d and project 92c49142ea324049ba24b55ff84e5a3b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:00.969388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:07:00.973565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:07:01.029543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-23b6441f-49b4-4b8c-83ee-fb6afbbe385e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.385038 May 19 12:07:01.030707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 90/194] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:06:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1388 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:07:01.041847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7d9c0238-b96a-4137-801b-495b09fd6b81 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:01.042654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7d9c0238-b96a-4137-801b-495b09fd6b81 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:01.143596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7d9c0238-b96a-4137-801b-495b09fd6b81 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:01.152093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7d9c0238-b96a-4137-801b-495b09fd6b81 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1217 microversion: 2.1 time: 0.115134 May 19 12:07:01.156915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 105/195] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:01 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1217 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:01.224078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:01.225053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:01.239520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:01.239984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:01.242507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:01.245306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:01.605173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:01.633724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6cf992c0-867a-4ba5-8a74-a0bec887108d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.411086 May 19 12:07:01.638004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 91/196] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:01 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:02.169651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:02.170300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:02.177332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:02.177992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:02.182517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:02.182940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:02.282977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:02.295114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38ac2ddb-45fe-4b49-961c-1cbbfe9f8ca5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1221 microversion: 2.1 time: 0.129350 May 19 12:07:02.300744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 106/197] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:02 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1221 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:02.655239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:02.655995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:02.663897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:02.664682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:02.665450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:02.665891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:02.829862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:02.838353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-961829a4-7411-4e31-a0b3-c36c3fdde653 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.187578 May 19 12:07:02.848501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 92/198] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:02 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:03.312177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:03.312918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:03.319516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:03.319952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:03.320735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:03.321319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:03.398591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:03.408356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-668ec75f-1e3f-4d68-84db-3aa92a76b084 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1221 microversion: 2.1 time: 0.099889 May 19 12:07:03.409051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 107/199] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:03 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1221 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:03.855402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:03.856254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:03.862739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:03.863184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:03.863905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:03.864329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:04.005473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:04.013695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-52a72d28-458e-4f6b-aad7-93ba5152b028 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1522 microversion: 2.1 time: 0.162300 May 19 12:07:04.023410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 93/200] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:03 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:04.426402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:04.427469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:04.434497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:04.435139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:04.436054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:04.436798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:04.518707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:04.527179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9ef7c49f-3998-498f-9e28-7a10e95cb740 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1221 microversion: 2.1 time: 0.105519 May 19 12:07:04.536581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 108/201] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:04 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1221 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:05.031244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:05.031901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:05.040210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.041330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.042163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.043016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.200675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:05.214032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0765879a-3976-4e7f-be95-060379f49fdc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1502 microversion: 2.1 time: 0.186494 May 19 12:07:05.231541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:05.232873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:05.234940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 94/202] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:05 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1502 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:05.245693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.246169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.246944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.247361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.496362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1779602416", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:05.555300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] validate_networks() for None {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:07:05.750015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:05.760215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a3bee408-35b0-4002-bd69-950419c9fb1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1515 microversion: 2.1 time: 0.532886 May 19 12:07:05.764214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 109/203] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:05 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1515 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:05.769481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:05.770251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:05.777195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.777811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.786610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.787629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.790945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:05.951116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:07:05.951616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:07:05.952150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:07:05.952700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:07:05.953195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:07:05.974577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.974831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.975865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:05.976457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:05.994947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Getting quotas for project 99c5725383e84ebea4a4aaed91598a18. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:05.999192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Getting quotas for user 3ec3996d0f6049b5afbe3b58eb232225 and project 99c5725383e84ebea4a4aaed91598a18. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:06.016548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:07:06.019899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:07:06.043196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:06.056127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-163fea00-bc85-4b31-b9e2-3ea7228ba490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1287 microversion: 2.1 time: 0.290342 May 19 12:07:06.058581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 110/204] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:05 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1287 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:06.110171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e7130bdb-a5ef-4910-b2f3-38280c24cc83 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.617652 May 19 12:07:06.112122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 95/205] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:07:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 621 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:07:06.122702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-493581cd-0b68-4e49-8957-d8a40038be36 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:06.124528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-493581cd-0b68-4e49-8957-d8a40038be36 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:06.228729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-493581cd-0b68-4e49-8957-d8a40038be36 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:06.248480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-493581cd-0b68-4e49-8957-d8a40038be36 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1215 microversion: 2.1 time: 0.129963 May 19 12:07:06.250791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 111/206] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:06 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1215 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:07.076564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:07.077211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:07.095591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.096357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.097324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.105523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.230880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:07.253519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0b19cbb-795a-4854-b08b-6e70ea06a3cd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1275 microversion: 2.1 time: 0.182144 May 19 12:07:07.256906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 96/207] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:07 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1275 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:07.268032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:07.269635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:07.280364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.280844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.281668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.282475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.382587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:07.391697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-14fb5df2-3f1b-461c-a92f-e191880ebf7b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1267 microversion: 2.1 time: 0.128586 May 19 12:07:07.393304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 112/208] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:07 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1267 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:07.428294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:07.429660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:07.436544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.437565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.438621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.439352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.484931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'flavor' on Instance uuid da2ea452-9604-40b7-b9cb-65be5fa21be5 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:07:07.659001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-384abecf-ee96-4f49-a499-106d19b6fc04 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/action" status: 200 len: 32 microversion: 2.1 time: 0.233912 May 19 12:07:07.659578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 97/209] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:07:07 2022] POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/action => generated 32 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:07:07.669535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:07.671071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:07.678928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.679579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:07.681018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:07.681767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:08.024246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:08.034965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-180ae93f-03f6-43cf-9444-8a07be6aa921 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.369759 May 19 12:07:08.047588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 113/210] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:07 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:08.272705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:08.274321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:08.295829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:08.296751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:08.297795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:08.298602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:08.409660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:08.410268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:08.428332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:08.428814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:08.429591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:08.430024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:08.515825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:08.524396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1dcb31c7-c5cf-45a2-ace7-223b86e4cd60 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.256530 May 19 12:07:08.525519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 98/211] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:08 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:08.692772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:08.702600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2088f834-d929-45cc-9019-52d69215c8f7 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1267 microversion: 2.1 time: 0.296739 May 19 12:07:08.706323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 114/212] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:08 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1267 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:09.059304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:09.060567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:09.069373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.070424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.071892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.072591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.237501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:09.249216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f11ec6b8-c703-4cd1-a51a-e886dbcd00a3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.193869 May 19 12:07:09.250404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 99/213] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:09 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:09.540735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:09.541879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:09.549173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.549804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.551311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.551888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.728859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:09.730198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:09.738970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.739802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.740871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:09.741876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:09.846319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:09.857927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f412346e-d13e-41bc-9fd9-631dfb6bf99b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1267 microversion: 2.1 time: 0.139976 May 19 12:07:09.861108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 100/214] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:09 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1267 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:09.926815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:09.939728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-88cd6718-9bb0-4b34-b8a4-8c9d137f5c1a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.402752 May 19 12:07:09.941182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 115/215] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:09 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:10.268063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:10.268586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:10.276052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.276793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:10.277569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.278155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:10.443350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:10.456251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a6b9499f-d796-48d5-8a4c-f40bc01f6372 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.192316 May 19 12:07:10.460680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 101/216] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:10 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:10.877072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:10.877691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:10.901790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.902736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:10.904069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.905463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:10.957681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:10.959615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:10.974906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.983466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:10.985417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:10.989949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:11.117702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:11.139657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b55c84ad-bdda-40d3-a961-c16afb5ea554 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1285 microversion: 2.1 time: 0.266810 May 19 12:07:11.147935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 116/217] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:10 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1285 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:11.429127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:11.437092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c07d3224-f0b9-428a-9d45-659d5984f952 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.483754 May 19 12:07:11.437690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 102/218] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:10 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:11.473838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:11.474453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:11.482720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:11.485719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:11.487492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:11.490746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:11.666099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d4d6018b-d0e3-4a46-b738-7c8348cbfe99 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Action: 'create', calling method: .version_select at 0x7fe20c2cb5e0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-150760622", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGAHnlZzKQqmb/aoQLhEdcCzNZeKcSLVZGtHQl/PrECrpKLyIfzLUnPxI41MeDoq5RgIAf4sboCm9b038n86JdxC1l8uPK83JSibpvpCRNIjHGq0z6v+sssYmQ84OrejFg=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:11.683603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-d4d6018b-d0e3-4a46-b738-7c8348cbfe99 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Getting quotas for project b7ffaf0276394eafb5a4ee10ef3ab4d7. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:11.689629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-d4d6018b-d0e3-4a46-b738-7c8348cbfe99 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Getting quotas for user 19ccb0e5ef064cfda61c913034da54a6 and project b7ffaf0276394eafb5a4ee10ef3ab4d7. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:11.719681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d4d6018b-d0e3-4a46-b738-7c8348cbfe99 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.327632 May 19 12:07:11.721518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 117/219] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:07:11 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:07:11.766152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:11.779056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a4aa3d96-9fb0-4dc0-b4fb-f783a6d0a3a6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.309023 May 19 12:07:11.779659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 103/220] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:11 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:12.163513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:12.164385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:12.179194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.179640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.180537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.184251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.454631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:12.455238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:12.462498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.462938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.463702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.468134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.525230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:12.555514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-989514b8-246e-402a-91f6-be980ff6c0e3 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.395918 May 19 12:07:12.556383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 118/221] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:12 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:12.689320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-c8471367-c60c-48d4-ba4a-dce211786a25 req-0424a75a-c327-4b62-9902-9d6f96ae429d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "182ed917-1ffe-42f8-9af3-65bcb94ae5c4", "tag": "f38c2817-0741-4776-a66b-c6da3da4ed42"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:12.713324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-c8471367-c60c-48d4-ba4a-dce211786a25 req-0424a75a-c327-4b62-9902-9d6f96ae429d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.715604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-c8471367-c60c-48d4-ba4a-dce211786a25 req-0424a75a-c327-4b62-9902-9d6f96ae429d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.744543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-c8471367-c60c-48d4-ba4a-dce211786a25 req-0424a75a-c327-4b62-9902-9d6f96ae429d service nova] Creating event network-changed:f38c2817-0741-4776-a66b-c6da3da4ed42 for instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 on hv-cinder-80403 May 19 12:07:12.754110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-c8471367-c60c-48d4-ba4a-dce211786a25 req-0424a75a-c327-4b62-9902-9d6f96ae429d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068764 May 19 12:07:12.755248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 119/222] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:07:12.781168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:12.794786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b2e13496-5216-44cb-b860-99d7f88b89a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.344499 May 19 12:07:12.796567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 104/223] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:12 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:12.796762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:12.797546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:12.822110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.825643 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:12.827447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:12.829964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:13.163890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:13.174366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8eea73d9-c0b8-4283-97c6-0fa6acfe4196 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.382268 May 19 12:07:13.188425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 120/224] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:12 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:13.572949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:13.574416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:13.594696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:13.595169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:13.595944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:13.596361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:13.603185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:13.813197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:13.814155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:13.823819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:13.828257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:13.830042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:13.832832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:14.088052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:14.106341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:14.107267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0d7af169-340e-42e0-9152-cc065173aa8e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.298528 May 19 12:07:14.108113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 121/225] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:13 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:14.114275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5664c304-884a-44b0-8b37-9900f4aba58a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.545518 May 19 12:07:14.114846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 105/226] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:13 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:14.191881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:14.192537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:14.206038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:14.206486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:14.207272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:14.207704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:14.215084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-343c897b-e517-4a59-a655-302930e49168 req-2d4d17f8-1331-4611-b2fc-8ef537937a24 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "75c37c27-474e-4b79-85ab-f983f5cb5815", "tag": "3fc3d1a1-a3ad-46b5-9781-c4f531e64600"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:14.251068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-343c897b-e517-4a59-a655-302930e49168 req-2d4d17f8-1331-4611-b2fc-8ef537937a24 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:14.251845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-343c897b-e517-4a59-a655-302930e49168 req-2d4d17f8-1331-4611-b2fc-8ef537937a24 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:14.267130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-343c897b-e517-4a59-a655-302930e49168 req-2d4d17f8-1331-4611-b2fc-8ef537937a24 service nova] Creating event network-changed:3fc3d1a1-a3ad-46b5-9781-c4f531e64600 for instance 75c37c27-474e-4b79-85ab-f983f5cb5815 on hv-cinder-80403 May 19 12:07:14.272835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-343c897b-e517-4a59-a655-302930e49168 req-2d4d17f8-1331-4611-b2fc-8ef537937a24 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061184 May 19 12:07:14.274151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 106/227] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:14.396452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:14.407083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f93ca56e-e5b4-4b1f-9004-bfc9f1127195 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.218696 May 19 12:07:14.411520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 122/228] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:14 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:15.125930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:15.127377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:15.131353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:15.132005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:15.135245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.136287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.137117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.137546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.139169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.139777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.141109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.141535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.277324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:15.287944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:15.294494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-46489a7b-35b3-4dd9-82ff-229ef54902ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.172990 May 19 12:07:15.295442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 107/229] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:15 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:15.307682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-725e6952-64c1-4aac-9da5-5857a8a8ca03 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.180326 May 19 12:07:15.308530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 123/230] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:15 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:15.429089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:15.430122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:15.437071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.438181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.439085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:15.439873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:15.585874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:15.593645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a6775d50-6c65-42bf-9be4-d54dbcb31f17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.168515 May 19 12:07:15.594218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 108/231] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:15 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:16.315246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:16.315860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:16.322717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.322897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.323619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.324045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.324748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:16.325349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:16.327944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:16.329011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=16 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:16.334940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.335372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.336302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.336902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.462487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:16.471007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62585e7e-5019-41fc-8a2e-e7c7a556ffd9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.161531 May 19 12:07:16.471582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 124/232] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:16 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:16.474041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:16.481757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dffd4ee3-4a00-44f1-a9e9-4e6c20dee52d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.161075 May 19 12:07:16.484728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 109/233] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:16 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:16.613680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:16.614285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:16.620664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.621129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.621882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:16.622306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:16.984747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:16.994303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07902d63-b92e-4acd-b8d5-378dd67c97cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.385388 May 19 12:07:16.995096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 125/234] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:16 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:17.487688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:17.488255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:17.495915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:17.496358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:17.497161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:17.497585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:17.498302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:17.498864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:17.507230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:17.508075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:17.508780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:17.509212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:17.644853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:17.652575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a5f6934f-aa03-4831-bf39-ca8297b35aa6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.168892 May 19 12:07:17.653264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 110/235] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:17 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:17.656069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:17.663980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6b475129-b966-4ec9-abb1-7fbb97fb7989 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.169628 May 19 12:07:17.664533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 126/236] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:17 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:18.014660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:18.015363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:18.022675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.023114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.023862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.024289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.157496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:18.165375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-99da4345-634d-409a-bc07-0837a13e262a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.157305 May 19 12:07:18.166044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 111/237] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:18 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:18.669655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:18.670253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:18.676485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.677152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.677964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.678539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.684388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:18.685666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:18.692980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.693781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.694556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:18.695004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:18.924960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:18.933315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:18.944477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c28a753a-44e2-41a9-8080-7d22cad516bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.278670 May 19 12:07:18.958123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 127/238] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:18 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:18.961098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4794d942-4965-4f7c-80d0-687eb6e7bfce tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.281534 May 19 12:07:18.961753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 112/239] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:18 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:19.184063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:19.184600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:19.191387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:19.191832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:19.192596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:19.193057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:19.334145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:19.341872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-94a58e8d-59e0-49b0-b7eb-9f58f7d4980c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.161878 May 19 12:07:19.342491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 128/240] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:19 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:19.708964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1741609102", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-246156379", "security_groups": [{"name": "tempest-secgroup-smoke-340134724"}], "networks": [{"uuid": "b6ed8ed4-3777-4a37-b2e0-507617c095d2"}], "block_device_mapping_v2": [{"uuid": "8fcd18a3-4226-4a1b-9a67-7f772c353244", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:19.838433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] validate_networks() for [('b6ed8ed4-3777-4a37-b2e0-507617c095d2', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:07:19.970075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:19.973222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:19.984656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:19.985243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:19.985916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:19.986344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.002673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:20.056517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=7 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:20.130477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:20.138921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f3f70687-3716-4c2a-bb7a-b1876f1e25c2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.177147 May 19 12:07:20.139515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 129/241] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:19 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:20.153124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:20.153741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:20.160130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.160661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.161513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.162000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.260803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:07:20.261226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:07:20.261694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:07:20.262091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:07:20.262495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:07:20.273349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.273788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.274572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.275002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.297091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Getting quotas for project 1082ac24dda74201ab7a9bf670d83a38. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:20.302177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Getting quotas for user f8e5131aab4e43a18d98e719e83d6268 and project 1082ac24dda74201ab7a9bf670d83a38. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:20.302871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:20.310597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa20679e-2026-46eb-8860-8cd2ae63b0cf tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.161293 May 19 12:07:20.311134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 130/242] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:20 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:20.359992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:20.360583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:20.365832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:07:20.367051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.367530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.367914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8fcd18a3-4226-4a1b-9a67-7f772c353244',volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:07:20.368301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:20.368756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:20.512107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c8176e7-c927-4f65-90c1-5fd2e9c5dac4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.806862 May 19 12:07:20.512786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 113/243] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:07:19 2022] POST /compute/v2.1/servers => generated 405 bytes in 809 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:07:20.513292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:20.523025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e7e26360-5cbd-4cee-881d-0a72f27a7a28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.167383 May 19 12:07:20.523332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db361d52-5919-4293-b681-4f931bd090b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:20.523876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 131/244] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:20 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:20.524469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db361d52-5919-4293-b681-4f931bd090b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:20.581721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-db361d52-5919-4293-b681-4f931bd090b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:20.588862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db361d52-5919-4293-b681-4f931bd090b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1088 microversion: 2.1 time: 0.069447 May 19 12:07:20.589897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 114/245] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1088 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:21.155606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:21.156312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:21.162623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.163161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.163907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.164326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.305282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:21.312851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8ca4a7f9-a68a-4fb3-aa71-153bdc1dad57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.161155 May 19 12:07:21.313424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 132/246] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:21 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:21.327721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:21.328447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:21.335496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.335935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.336684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.337114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.469809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:21.478097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fe1f5376-87da-4722-8002-777918e09f9b tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.154153 May 19 12:07:21.478700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 115/247] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:21 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:21.540666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:21.541296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:21.548579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.549008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.549753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.550202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.606816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:21.607413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:21.615738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.616161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.616968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:21.618186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:21.688315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:21.697922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8d0c8458-e69f-49f2-80bb-d1a389f4a024 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1186 microversion: 2.1 time: 0.095542 May 19 12:07:21.698519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 116/248] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1186 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:21.698932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:21.701344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=8 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:21.708182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58689b19-9c16-4dc2-8da6-44cb724e8c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.171386 May 19 12:07:21.708876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 133/249] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:21 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:22.332560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:22.333841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:22.342426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.343118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.344067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.345057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.495740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:22.496514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:22.504944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:22.506672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.509131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.509928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:22.511322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.512064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.520521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fd3e1210-443c-4f38-aa26-aedfa769e008 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.194226 May 19 12:07:22.521188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 117/250] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:22 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:22.719382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:22.720571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:22.730298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.731002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.732981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.733504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.748858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:22.758582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7f276afb-81fe-4dc1-a63c-213a41f3e072 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.267544 May 19 12:07:22.768812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 134/251] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:22 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:22.774866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:22.776109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:22.782988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.784139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.790130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:22.790883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:22.914907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:22.925552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c41f8ddf-3e36-44e3-a7bf-1ee37a0229b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1271 microversion: 2.1 time: 0.210783 May 19 12:07:22.927084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 118/252] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1271 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:22.995251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:23.006180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e91e08e3-6598-4a73-90f0-f491defe1753 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.245076 May 19 12:07:23.016991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 135/253] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:22 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:23.303989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-2badfa7b-2c37-4e6e-b795-c17b4eb7fad1 req-eeb60fee-3763-4d63-b6b3-824f787e56b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "182ed917-1ffe-42f8-9af3-65bcb94ae5c4", "name": "network-vif-plugged", "status": "completed", "tag": "f38c2817-0741-4776-a66b-c6da3da4ed42"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:23.317384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2badfa7b-2c37-4e6e-b795-c17b4eb7fad1 req-eeb60fee-3763-4d63-b6b3-824f787e56b7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.319121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2badfa7b-2c37-4e6e-b795-c17b4eb7fad1 req-eeb60fee-3763-4d63-b6b3-824f787e56b7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.343208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-2badfa7b-2c37-4e6e-b795-c17b4eb7fad1 req-eeb60fee-3763-4d63-b6b3-824f787e56b7 service nova] Creating event network-vif-plugged:f38c2817-0741-4776-a66b-c6da3da4ed42 for instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 on hv-cinder-80403 May 19 12:07:23.349874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-2badfa7b-2c37-4e6e-b795-c17b4eb7fad1 req-eeb60fee-3763-4d63-b6b3-824f787e56b7 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052732 May 19 12:07:23.350608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 119/254] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:23.539267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:23.540308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:23.552614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.553313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.554617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.555401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.788098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:23.789974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:23.807191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.807792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.809267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.809812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.840526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:23.851557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-465287e3-295a-4b93-894f-192ed5129f24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.316841 May 19 12:07:23.852839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 136/255] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:23 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:23.943300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:23.944162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:23.959720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.962020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:23.964940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:23.965882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:24.020765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:24.034908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-15b9da90-2855-4d37-bdbc-c4528206083a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.260556 May 19 12:07:24.035546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 120/256] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:23 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:24.040824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:24.042416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:24.050452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:24.051220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:24.052308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:24.052942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:24.217455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:24.231125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3ba96528-7994-4bd2-a429-1fb949c5e3c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.291504 May 19 12:07:24.243256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 137/257] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:24.248183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-713023db-b171-4210-b62b-210d8af3ca0c req-32e6279b-a44f-4169-af90-bf3390cbde34 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:24.263438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-713023db-b171-4210-b62b-210d8af3ca0c req-32e6279b-a44f-4169-af90-bf3390cbde34 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:24.264527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-713023db-b171-4210-b62b-210d8af3ca0c req-32e6279b-a44f-4169-af90-bf3390cbde34 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:24.293229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-713023db-b171-4210-b62b-210d8af3ca0c req-32e6279b-a44f-4169-af90-bf3390cbde34 service nova] Creating event network-changed:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:07:24.304156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-713023db-b171-4210-b62b-210d8af3ca0c req-32e6279b-a44f-4169-af90-bf3390cbde34 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059414 May 19 12:07:24.304803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 138/258] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:24.322893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:24.331526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4d6846bc-0f9a-495f-bf6f-31148443e822 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.294679 May 19 12:07:24.332113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 121/259] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:24 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:24.869778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:24.871071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:24.886231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:24.886994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:24.887989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:24.888501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.057504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:25.058630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:25.067566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.068180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.069292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.069862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.117670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:25.126224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a1044d76-8edb-418d-a6a8-8e75104e2453 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1317 microversion: 2.1 time: 0.260501 May 19 12:07:25.136386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 139/260] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:24 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1317 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:25.226020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:25.239631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-27514216-71d6-4306-949c-c4a6a2c7872d tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.186740 May 19 12:07:25.241101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 122/261] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:25 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:25.254203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:25.254815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:25.261581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.262031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.262816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.263316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.351942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:25.352764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:25.360267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.360747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.361653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.362082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.432588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:25.447452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f94dce77-7f99-4e81-9f4c-42f6a4e6829e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.200248 May 19 12:07:25.447954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 140/262] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:25.455969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-87ce906d-8fd4-4c8b-b92b-689b03f53b92 req-5f0cad25-f69a-4555-a31a-6495ab9aa92c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75c37c27-474e-4b79-85ab-f983f5cb5815", "name": "network-vif-plugged", "status": "completed", "tag": "3fc3d1a1-a3ad-46b5-9781-c4f531e64600"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:25.474922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-87ce906d-8fd4-4c8b-b92b-689b03f53b92 req-5f0cad25-f69a-4555-a31a-6495ab9aa92c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:25.477281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-87ce906d-8fd4-4c8b-b92b-689b03f53b92 req-5f0cad25-f69a-4555-a31a-6495ab9aa92c service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:25.490412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-87ce906d-8fd4-4c8b-b92b-689b03f53b92 req-5f0cad25-f69a-4555-a31a-6495ab9aa92c service nova] Creating event network-vif-plugged:3fc3d1a1-a3ad-46b5-9781-c4f531e64600 for instance 75c37c27-474e-4b79-85ab-f983f5cb5815 on hv-cinder-80403 May 19 12:07:25.511041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-87ce906d-8fd4-4c8b-b92b-689b03f53b92 req-5f0cad25-f69a-4555-a31a-6495ab9aa92c service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058727 May 19 12:07:25.511987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 141/263] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:25.520746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:25.528317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-97a9408a-8ce3-490e-be6a-6e938e847ae1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.182630 May 19 12:07:25.528953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 123/264] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:25 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:26.147662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:26.148997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:26.159064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.159539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.160922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.161873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.257000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:26.258320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:26.265278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.265838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.266710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.267228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.331772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:26.339736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2b698533-cca3-41c0-b8b1-0342548306ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1506 microversion: 2.1 time: 0.199175 May 19 12:07:26.340314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 142/265] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:26 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1506 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:26.466495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:26.467355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:26.476151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:26.482757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:26.482990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.483436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.484216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.484700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.486427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-87df4c9a-370b-4772-9b76-c46266e4d72a tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.233182 May 19 12:07:26.487725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 124/266] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:26 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:26.545304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:26.545903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:26.552659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.553320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.554316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:26.554811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:26.772513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:26.783228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee6c292d-891c-4315-850d-9b2e210af2b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.240736 May 19 12:07:26.784614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 125/267] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:26 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:26.786382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:26.803965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9f09723d-35df-45e2-a089-48142bba5bb7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.341246 May 19 12:07:26.813758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 143/268] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:27.508504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:27.512183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:27.519656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.520341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:27.521143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.521591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:27.665438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:27.673043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e6a71daa-6306-4c07-aa5c-ac6bd879ef73 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1315 microversion: 2.1 time: 0.170542 May 19 12:07:27.673609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 126/269] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:27 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:27.803392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:27.803971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:27.810229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.810667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:27.811424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.811852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:27.820722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:27.821520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:27.829126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.829724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:27.830847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:27.831450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:28.013910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:28.021933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ed82470-35fc-4787-949c-30929cc95adc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.204942 May 19 12:07:28.022715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 127/270] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:28.345188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:28.353911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8ce4d63e-4f45-450a-9f8c-694978b51986 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.554388 May 19 12:07:28.354778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 144/271] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:27 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:28.690938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:28.691518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:28.698231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:28.698777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:28.699414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:28.699831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:28.841036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:28.848627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0311f780-1477-43f8-89b6-f7ced8641646 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1465 microversion: 2.1 time: 0.162386 May 19 12:07:28.849747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 128/272] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:28 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1465 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:28.888207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:28.888785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:28.897768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:28.898477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:28.899240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:28.899692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:28.925071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:07:28.979961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:29.020929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9fdd0eca-c478-416e-ae00-c46d0bc89a62 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "DELETE /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 204 len: 0 microversion: 2.1 time: 0.136712 May 19 12:07:29.021515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 145/273] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:07:28 2022] DELETE /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:07:29.030414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:29.030942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:29.037877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.038325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.039131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.039512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.040153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:29.040533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:29.053460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.054038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.054815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.055459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.206558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:29.215982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e2ebe8c-ba7c-4e0a-b040-950a805b2a33 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1471 microversion: 2.1 time: 0.188676 May 19 12:07:29.216604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 129/274] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:29 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1471 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:29.291358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:29.300989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07affc41-977c-4f64-9975-47cbddae38da tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.265879 May 19 12:07:29.301762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 146/275] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:29.371488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:29.372217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:29.379105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.379538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.380288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:29.380743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:29.528826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:29.536402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9946c5eb-6d24-46a4-bfca-2cbb8372a103 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.168837 May 19 12:07:29.536944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 130/276] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:29 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:30.233655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:30.234279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:30.240519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.240992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.241739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.242165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.318600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:30.321476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:30.323979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:30.329903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.330349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.331112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.331535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.381989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:30.389901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-efd3246d-7b3d-4849-8f9a-babb57038823 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1471 microversion: 2.1 time: 0.160128 May 19 12:07:30.390472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 147/277] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:30 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1471 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:30.465592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:30.474100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e3574d2a-b0df-4919-bcef-d64fb650c806 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.159366 May 19 12:07:30.474684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 131/278] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:30.553321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:30.554336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:30.561244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.561689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.562616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:30.563048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:30.705793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:30.713475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1826e3f2-cf2e-4bcf-b88a-e0d6023482b5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.164227 May 19 12:07:30.715077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 148/279] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:30 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:31.407971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:31.408717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:31.411152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=8 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:31.417772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.418178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.419026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.419465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.491826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:31.493066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:31.505448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.507138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.508496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.510214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.575598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:31.585214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e38674d6-b5bf-4fea-878e-6aa833d92864 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1471 microversion: 2.1 time: 0.181106 May 19 12:07:31.585816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 132/280] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:31 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1471 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:31.724924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:31.732805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-66197e8c-0f43-4d7b-8265-ea4a24fc77d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.245046 May 19 12:07:31.734349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 149/281] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:31.743453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:31.744064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:31.752169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.753104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.754222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.754961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.897379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:31.909732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-01c1e678-2dda-400d-b1d9-95802379e528 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.178608 May 19 12:07:31.913834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 133/282] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:31 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:31.940385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-cf4b0734-0058-4d2a-93ca-21e1f1e18920 req-69ba73ce-3f2f-4bea-8d87-4df2aa67e9f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75c37c27-474e-4b79-85ab-f983f5cb5815", "name": "network-vif-deleted", "tag": "3fc3d1a1-a3ad-46b5-9781-c4f531e64600"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:31.952887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-cf4b0734-0058-4d2a-93ca-21e1f1e18920 req-69ba73ce-3f2f-4bea-8d87-4df2aa67e9f5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:31.953400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-cf4b0734-0058-4d2a-93ca-21e1f1e18920 req-69ba73ce-3f2f-4bea-8d87-4df2aa67e9f5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:31.971839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-cf4b0734-0058-4d2a-93ca-21e1f1e18920 req-69ba73ce-3f2f-4bea-8d87-4df2aa67e9f5 service nova] Creating event network-vif-deleted:3fc3d1a1-a3ad-46b5-9781-c4f531e64600 for instance 75c37c27-474e-4b79-85ab-f983f5cb5815 on hv-cinder-80403 May 19 12:07:31.978893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-cf4b0734-0058-4d2a-93ca-21e1f1e18920 req-69ba73ce-3f2f-4bea-8d87-4df2aa67e9f5 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042658 May 19 12:07:31.982422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 150/283] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:07:32.608132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:32.608768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:32.616499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.616963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:32.617723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.618234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:32.684698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec3996d0f6049b5afbe3b58eb232225', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '99c5725383e84ebea4a4aaed91598a18', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:32.692360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cb92e870-615e-4d24-98b7-6f88fd9452af tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 200 len: 1429 microversion: 2.1 time: 0.092865 May 19 12:07:32.692969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 134/284] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:32 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 1429 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:32.749972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:32.750663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:32.757191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.757627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:32.758480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.758969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:32.903335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:32.913359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dbe9fc7c-246e-46b8-89fc-61b015d48831 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.167446 May 19 12:07:32.914518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 151/285] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:32.931523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:32.932164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:32.939204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.939793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:32.940553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:32.941108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.077562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:33.085483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c3bd42f8-1acc-4bc5-ac56-f45c767e9654 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.158676 May 19 12:07:33.086031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 135/286] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:32 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:33.627407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:33.627885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:33.636313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.637008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.638026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.638859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.670708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0ee77d24-c8a3-4c5c-ae50-7a1f98410ff5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/ips" status: 200 len: 148 microversion: 2.20 time: 0.047121 May 19 12:07:33.672668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 152/287] 10.222.0.35 () {60 vars in 1288 bytes} [Thu May 19 12:07:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/ips => generated 148 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:07:33.710154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:33.712931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] [instance: 75c37c27-474e-4b79-85ab-f983f5cb5815] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:33.722779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.723255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.724025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.724455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.738637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] HTTP exception thrown: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Request method failure captured: May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 HTTP/1.1 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLk-hqg9c4n4ceePERRdNWH64ZzeKS44q-984aJpz--PcsEQXXUOaI3CIB4bxenjJz4mQR2pfqq0vuAEsyC2kN_NK9bznl2NAgCIOKMaFy6giXi4LKGqC74wicxRk0VJ7nqYV0vThgfGu5S9GA5iSXvOy4wbyOM6qJUhZHxdPCaPwY May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 99c5725383e84ebea4a4aaed91598a18 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesNegativeTest-1466111896 May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:07:33.740438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/99c5725383e84ebea4a4aaed91598a18"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/99c5725383e84ebea4a4aaed91598a18"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_99c5725383e84ebea4a4aaed91598a18"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/99c5725383e84ebea4a4aaed91598a18"}]}] May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesNegativeTest-1466111896 May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 99c5725383e84ebea4a4aaed91598a18 May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesNegativeTest-1466111896 May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesNegativeTest-1466111896-project May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 3ec3996d0f6049b5afbe3b58eb232225 May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesNegativeTest-1466111896-project May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. May 19 12:07:33.742055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '75c37c27-474e-4b79-85ab-f983f5cb5815'} May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:07:33.742904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. May 19 12:07:33.744154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:33.747329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] Returning 404 to user: Instance 75c37c27-474e-4b79-85ab-f983f5cb5815 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:07:33.747329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ba933f42-5ff7-419e-946c-6fb75fff8172 tempest-VolumesNegativeTest-1466111896 tempest-VolumesNegativeTest-1466111896-project] 10.222.0.35 "GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815" status: 404 len: 111 microversion: 2.1 time: 0.036387 May 19 12:07:33.748937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 136/288] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:33 2022] GET /compute/v2.1/servers/75c37c27-474e-4b79-85ab-f983f5cb5815 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:07:33.935300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:33.936279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:33.944007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.944826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:33.946360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:33.946985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:34.103431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:34.104707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:34.112360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:34.112921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:34.113847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:34.114694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:34.206041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:34.215133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fdf3c274-261f-4997-a250-48668e13de06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.288043 May 19 12:07:34.224095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 153/289] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:34.310335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:34.320352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-540218cb-e296-4d51-856e-93d4ea04fbd5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.221695 May 19 12:07:34.321416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 137/290] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:34 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:35.231597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:35.232800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:35.240601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:35.241108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:35.245609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:35.246050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:35.345229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:35.345846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:35.360668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:35.361161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:35.361906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:35.362365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:35.487726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:35.497928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a77faad5-2ca8-4e2a-bf77-18af7804c517 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.270456 May 19 12:07:35.498494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 154/291] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:35.579017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:35.588593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ef32a50-3317-479b-a1e5-6a54b002a4b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.250733 May 19 12:07:35.589216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 138/292] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:35 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:36.518066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:36.519501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:36.527431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:36.528196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:36.530778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:36.531795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:36.606304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:36.606991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:36.613534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:36.613969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:36.614730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:36.615156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:36.696811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:36.704571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-03911607-a242-497d-bc6d-2d7d94aa3c83 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.190624 May 19 12:07:36.705251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 155/293] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:36.797204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:36.806552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a108910-8a29-45b4-8629-2bf65bd14955 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.203127 May 19 12:07:36.807158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 139/294] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:36 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:37.365687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-41c3a19b-e2f1-43de-82ad-d3acd7dfc69a req-7699711c-84fe-4680-baff-19c32b1de617 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "name": "network-vif-plugged", "status": "completed", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:37.382313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-41c3a19b-e2f1-43de-82ad-d3acd7dfc69a req-7699711c-84fe-4680-baff-19c32b1de617 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:37.382717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-41c3a19b-e2f1-43de-82ad-d3acd7dfc69a req-7699711c-84fe-4680-baff-19c32b1de617 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:37.395012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-41c3a19b-e2f1-43de-82ad-d3acd7dfc69a req-7699711c-84fe-4680-baff-19c32b1de617 service nova] Creating event network-vif-plugged:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:07:37.400536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-41c3a19b-e2f1-43de-82ad-d3acd7dfc69a req-7699711c-84fe-4680-baff-19c32b1de617 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040423 May 19 12:07:37.401227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 156/295] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:37.723360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:37.724132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:37.732044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:37.732491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:37.733662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:37.734844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:37.824391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:37.825185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:37.831911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:37.832359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:37.833147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:37.833568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:37.892440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:37.902123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e9f2323-ae85-4093-b09b-c46244856d0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.183454 May 19 12:07:37.902783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 140/296] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:37.968035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:37.976539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c11da28b-9386-4481-87ec-d70a9a7b064e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.156567 May 19 12:07:37.977448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 157/297] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:37 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:38.494167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:38.494655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:38.501282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.501630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.502850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.502850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.530574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3fcfa349-39a8-40c4-a345-cb89f85734b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/ips" status: 200 len: 144 microversion: 2.1 time: 0.040180 May 19 12:07:38.531439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 141/298] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:07:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/ips => generated 144 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:38.876561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:38.877211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:38.886651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.887112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.887950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.888315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.917570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lazy-loading 'flavor' on Instance uuid b642ea31-4427-475b-8374-591f4dcbadc8 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:07:38.919390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:38.920090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:38.928594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.929092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.929860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:38.930566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:38.947707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:39.101088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:39.109896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-52970baf-8a5a-4281-82d4-0a2ec8cf6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.195068 May 19 12:07:39.110522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 142/299] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:39.116449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:39.121219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:39.128802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.129434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.130351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.130940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.312951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:39.320567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-090a3e68-b967-470d-8a4f-bd056fceec01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1501 microversion: 2.1 time: 0.208902 May 19 12:07:39.321212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 143/300] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:39 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1501 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:39.339119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6580e2de-0671-48b5-a261-bb5694127da8"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:39.340272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:39.346993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.352042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.353186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.353627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.377515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Cannot 'attach_volume' instance da2ea452-9604-40b7-b9cb-65be5fa21be5 while it is in vm_state rescued May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/os-volume_attachments HTTP/1.1 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 74 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:07:39.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {"volumeAttachment": {"volumeId": "6580e2de-0671-48b5-a261-bb5694127da8"}} May 19 12:07:39.380506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Cannot 'attach_volume' instance da2ea452-9604-40b7-b9cb-65be5fa21be5 while it is in vm_state rescued May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'da2ea452-9604-40b7-b9cb-65be5fa21be5', 'body': {'volumeAttachment': {'volumeId': '6580e2de-0671-48b5-a261-bb5694127da8'}}} May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 in vm_state rescued. Cannot attach_volume while the instance is in this state. May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:07:39.381348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance da2ea452-9604-40b7-b9cb-65be5fa21be5 while it is in vm_state rescued May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:07:39.382522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance da2ea452-9604-40b7-b9cb-65be5fa21be5 while it is in vm_state rescued May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 409 to user: Cannot 'attach_volume' instance da2ea452-9604-40b7-b9cb-65be5fa21be5 while it is in vm_state rescued {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8d4c59ed-f52d-4e02-bfb6-65a7edb3c039 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.046992 May 19 12:07:39.384714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 144/301] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:07:39 2022] POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/os-volume_attachments => generated 152 bytes in 48 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) May 19 12:07:39.389753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:39.390253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:39.396840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.397283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.398053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.398482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.473680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ce041021-fea1-4b59-aaa0-147ddad745b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/action" status: 202 len: 0 microversion: 2.1 time: 0.087356 May 19 12:07:39.474370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 145/302] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:07:39 2022] POST /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 19 12:07:39.482930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:39.483614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:39.494327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.494807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.495644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:39.496068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:39.635974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:39.643918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e5f6073f-2f1c-4b4c-aaea-55e935ffecfd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.164862 May 19 12:07:39.644676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 146/303] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:39 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:40.126628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:40.127285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:40.137291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:40.137291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:40.138004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:40.138430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:40.300584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:40.315410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f8ae601c-0c32-41bf-b668-e9e28e573227 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1259 microversion: 2.1 time: 0.192492 May 19 12:07:40.315996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 147/304] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1259 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:40.662748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:40.663508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:40.670857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:40.671389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:40.672314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:40.672733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:40.814992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:40.823803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c29b89d4-f68b-464d-9204-6eff0fba8ca1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.165818 May 19 12:07:40.824467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 148/305] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:40 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:41.333284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:41.334056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:41.340894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.341333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:41.342235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.342656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:41.482399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:41.490410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8374a1f5-7ea7-4922-8275-0e43bcf0e351 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1442 microversion: 2.1 time: 0.162879 May 19 12:07:41.491030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 149/306] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1442 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:41.507995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:41.508594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:41.516067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.516497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:41.517372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.517792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:41.657949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:41.665746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2d905d54-bcd2-4de1-91f8-fb57fdc6af53 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1442 microversion: 2.1 time: 0.161847 May 19 12:07:41.666341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 150/307] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1442 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:41.841252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:41.841616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:41.848235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.848725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:41.849545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:41.849976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:42.110652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:42.119407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-546da5d1-d44e-46db-ac38-e49d3cd4cbb3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.282236 May 19 12:07:42.119973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 151/308] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:41 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:42.549848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1875442953", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-150760622", "security_groups": [{"name": "tempest-secgroup-smoke-1576464253"}], "block_device_mapping_v2": [{"uuid": "90eae458-b221-4272-bcb5-b75543a2e25a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "8710bde6-d1d0-413f-84e4-22dc9860558b"}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:42.974538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] validate_networks() for [('8710bde6-d1d0-413f-84e4-22dc9860558b', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:07:43.641319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:07:43.641907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:07:43.642175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:07:43.642608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:07:43.643196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:07:43.652824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:43.653506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:43.654312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:43.655152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:43.668293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:43.676079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Getting quotas for project b7ffaf0276394eafb5a4ee10ef3ab4d7. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:43.680835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Getting quotas for user 19ccb0e5ef064cfda61c913034da54a6 and project b7ffaf0276394eafb5a4ee10ef3ab4d7. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:43.744901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:07:43.747173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='90eae458-b221-4272-bcb5-b75543a2e25a',volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:07:43.898145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65f195e4-1882-46ee-9f01-7b152f6ba11e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.352194 May 19 12:07:43.903634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 152/309] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:07:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 1354 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:07:43.903634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:43.903865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:43.910688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:43.911127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:43.911907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:43.912326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:44.146143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:44.155910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa01a3c3-8931-469f-90b9-844b39b77a19 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.256098 May 19 12:07:44.156487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 153/310] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:43 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:44.162017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-060757cb-087a-4ab3-a116-57feefe26812 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:44.162916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-060757cb-087a-4ab3-a116-57feefe26812 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:44.261579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-060757cb-087a-4ab3-a116-57feefe26812 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:44.270919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-060757cb-087a-4ab3-a116-57feefe26812 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1094 microversion: 2.1 time: 0.112973 May 19 12:07:44.271540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 154/311] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:44 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1094 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:44.353491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-d4c3f6f2-060a-4c4d-88c3-ee467c465032 req-0d296cdd-e36a-458e-a5e1-762f23cf562f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:44.364607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-d4c3f6f2-060a-4c4d-88c3-ee467c465032 req-0d296cdd-e36a-458e-a5e1-762f23cf562f service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:44.365119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-d4c3f6f2-060a-4c4d-88c3-ee467c465032 req-0d296cdd-e36a-458e-a5e1-762f23cf562f service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:44.377431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-d4c3f6f2-060a-4c4d-88c3-ee467c465032 req-0d296cdd-e36a-458e-a5e1-762f23cf562f service nova] Creating event network-changed:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:07:44.382668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-d4c3f6f2-060a-4c4d-88c3-ee467c465032 req-0d296cdd-e36a-458e-a5e1-762f23cf562f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034634 May 19 12:07:44.383249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 155/312] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:45.174324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:45.175053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:45.185334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:45.185897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:45.186968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:45.187548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:45.337018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:45.345537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36bd1711-039c-463a-a906-b755ace118ee tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.175446 May 19 12:07:45.346122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 156/313] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:45 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:45.350873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:45.351467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:45.358017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:45.358484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:45.359247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:45.359681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:45.422394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:45.430607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4717a511-cb3e-4714-a812-35a8dc26c6c7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1144 microversion: 2.1 time: 0.083508 May 19 12:07:45.431197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 157/314] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:45 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1144 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:46.364966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:46.365660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:46.375264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:46.375757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:46.376938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:46.377538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:46.518434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:46.526673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-06228234-858a-4b03-b39d-de77f1b11935 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1509 microversion: 2.1 time: 0.166611 May 19 12:07:46.527212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 158/315] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:46 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1509 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:46.531889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:46.532436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:46.538309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:46.538748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:46.539494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:46.539919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:46.608325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:46.617104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-48be3cbd-c5e1-451e-95ea-2c3b2389a1ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1144 microversion: 2.1 time: 0.088817 May 19 12:07:46.617609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 159/316] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:46 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1144 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:47.546589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:47.547290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:47.556505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.556918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:47.557671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.558085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:47.703899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:47.706396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:47.713008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f14cf8d-eb5c-4122-adee-b9e1aa5b1f93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1515 microversion: 2.1 time: 0.172150 May 19 12:07:47.713574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 160/317] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:47 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1515 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:47.718542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:47.719134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:47.729088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.729088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:47.729088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.729088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:47.791765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:47.800849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-354a0f7e-a42b-4edd-8c9a-ab9eaea53a65 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1192 microversion: 2.1 time: 0.085919 May 19 12:07:47.801504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 161/318] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:47 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1192 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:47.873839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6f43fdb8-cc39-4d84-bce4-ec8a02bfa85b"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:47.874930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:47.881520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.882115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:47.883027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:47.884262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:48.500535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3311759-6db4-4b49-84d8-f3e9eddda256 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.630759 May 19 12:07:48.505723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 162/319] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:07:47 2022] POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 194 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:48.819001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:48.819693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:48.833803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:48.834241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:48.835226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:48.835732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:48.886888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:48.921308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:48.929312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-204dd287-e1ac-4408-a265-95fca0f34ebe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1210 microversion: 2.1 time: 0.114475 May 19 12:07:48.930212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 163/320] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:48 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1210 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:49.016230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:INSERT=2,nova_cell1:SELECT=77,nova_cell1:UPDATE=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:07:49.108960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45eba088-f04d-4c8b-b0d8-2948314d34e2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: .version_select at 0x7fe20c1f8dc0>, body: {"keypair": {"name": "tempest-keypair-1886018830", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEYc+eCgRLr54OS+N0eSq3qHp9ZVxDto1emDd7Tl2DWycn/f8bNw3EDrYtqhW8mnPvq6VHpj7Wj5J8kvpcG+nVgVnVgbMWv1vNBJTfUVjOw42qikjUAfBSex61iA93+Q2w=="}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:49.129254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-45eba088-f04d-4c8b-b0d8-2948314d34e2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for project 0ce848ee51f841638902e08ea1c2af28. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:49.134046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-45eba088-f04d-4c8b-b0d8-2948314d34e2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for user 36415049be5d4238a0275db21a8f378a and project 0ce848ee51f841638902e08ea1c2af28. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:49.171613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45eba088-f04d-4c8b-b0d8-2948314d34e2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.067210 May 19 12:07:49.172491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 164/321] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:07:49 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:07:49.948235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:49.949119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:49.957466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:49.957947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:49.958798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:49.959386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:50.176683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:50.188220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dea3b989-ba12-4897-8a85-a8341d18521f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.244424 May 19 12:07:50.197581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 165/322] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:49 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:07:50.498446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-2b07d38f-734d-4aed-b2ea-6a7fbf37c490 req-edc35a41-38e4-494f-aaae-d6facceab4aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7b377f1-b40c-4146-8569-44510a0c33ab", "tag": "1bd50345-4513-45c1-ae8b-577dff600ff2"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:50.518761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2b07d38f-734d-4aed-b2ea-6a7fbf37c490 req-edc35a41-38e4-494f-aaae-d6facceab4aa service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:50.519430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2b07d38f-734d-4aed-b2ea-6a7fbf37c490 req-edc35a41-38e4-494f-aaae-d6facceab4aa service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:50.553583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-2b07d38f-734d-4aed-b2ea-6a7fbf37c490 req-edc35a41-38e4-494f-aaae-d6facceab4aa service nova] Creating event network-changed:1bd50345-4513-45c1-ae8b-577dff600ff2 for instance a7b377f1-b40c-4146-8569-44510a0c33ab on hv-cinder-80403 May 19 12:07:50.562738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-2b07d38f-734d-4aed-b2ea-6a7fbf37c490 req-edc35a41-38e4-494f-aaae-d6facceab4aa service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068112 May 19 12:07:50.565089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 166/323] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:51.081025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2023599602", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3c40f460-44ee-4118-b5ce-daffc19a643a"}], "security_groups": [{"name": "tempest-securitygroup--2068752158"}], "key_name": "tempest-keypair-1886018830", "user_data": "}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:51.921205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] validate_networks() for [('3c40f460-44ee-4118-b5ce-daffc19a643a', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:07:51.949746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cc2a63ab-314c-48df-8e98-56c23a7a77b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/action" status: 202 len: 0 microversion: 2.20 time: 13.076004 May 19 12:07:51.950408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 158/324] 10.222.0.35 () {62 vars in 1318 bytes} [Thu May 19 12:07:38 2022] POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/action => generated 0 bytes in 13078 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 19 12:07:51.959909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:51.960608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:51.967215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:51.968224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:51.968583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:51.969096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:52.121128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:52.128899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-25f39a41-651d-4db4-b53e-c577930d9257 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.174326 May 19 12:07:52.129486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 159/325] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:51 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:52.139101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:52.139101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:52.144278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:52.144798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:52.145572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:52.145997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:52.288771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:52.300237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:52.303814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:52.305619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e9fce78b-62ad-47c6-910d-fdc8f064762f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.175008 May 19 12:07:52.306438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 160/326] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:52.525589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:07:52.526159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:07:52.527136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:07:52.527527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:07:52.527977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:07:52.537521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:52.537961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:52.538781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:52.539253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:52.556096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:07:52.560519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Getting quotas for user 36415049be5d4238a0275db21a8f378a and project 0ce848ee51f841638902e08ea1c2af28. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:07:52.571556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:07:52.574517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:07:52.619782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be9a02f6-63a0-4373-9973-cc24d942104b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.543782 May 19 12:07:52.624258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 167/327] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:07:51 2022] POST /compute/v2.1/servers => generated 409 bytes in 1545 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:07:52.630464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-29ed7798-724b-45fc-999e-36a32222a8aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:52.631065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-29ed7798-724b-45fc-999e-36a32222a8aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:52.693315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-29ed7798-724b-45fc-999e-36a32222a8aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:52.700127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-29ed7798-724b-45fc-999e-36a32222a8aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1246 microversion: 2.1 time: 0.073564 May 19 12:07:52.700750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 161/328] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1246 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:53.146268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:53.146877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:53.154026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.154469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.155219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.155649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.290776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:53.298629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b794435-bc4a-4042-8e71-71f12ad562d0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.156911 May 19 12:07:53.299182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 168/329] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:53 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:53.325537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:53.326291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:53.332663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.333111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.333865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.334284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.468671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:53.477068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:53.478565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:53.479725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19c29e71-a2f0-446d-be01-823c81efebc7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.158393 May 19 12:07:53.480285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 162/330] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:53.718247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:53.718920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:53.726684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.727367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.728034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:53.728795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:53.796307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:53.805764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-059e0528-01e1-44f7-84c1-9f2994bee623 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1298 microversion: 2.1 time: 0.091996 May 19 12:07:53.806593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 169/331] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1298 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:54.315526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:54.316121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:54.322402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.322841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.323673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.324087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.478349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:54.486502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c2de5f50-bf88-426e-a9f6-ace2ef87ad41 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.174881 May 19 12:07:54.487074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 163/332] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:54 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:54.498644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:54.499037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:54.508042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.508860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.509643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.510297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.728164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:54.737624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:54.739094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:54.740258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dabb059d-4d27-4b75-8a5b-ac5534784e09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.246329 May 19 12:07:54.741037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 170/333] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:54.823558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:54.824156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:54.830286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.830723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.831462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:54.831878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:54.992039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:55.001502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ce06b1ec-776e-485a-bc07-0cf9f2a8b712 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.181705 May 19 12:07:55.002092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 164/334] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:55.503975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:55.504570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:55.513773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:55.516407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:55.517202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:55.517629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:55.731335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:55.739825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ad329818-5bae-4a67-a830-b63eeb440c43 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.239475 May 19 12:07:55.740295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 171/335] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:55 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:55.762614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-9e9a2c45-bf29-454c-925e-11234419bc74 req-2fd89610-aa7b-4ed3-82f1-a3037aeb3681 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:55.763213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:55.763326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:55.772921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:55.773615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:55.775317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:55.775933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:55.781022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-9e9a2c45-bf29-454c-925e-11234419bc74 req-2fd89610-aa7b-4ed3-82f1-a3037aeb3681 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:55.781800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-9e9a2c45-bf29-454c-925e-11234419bc74 req-2fd89610-aa7b-4ed3-82f1-a3037aeb3681 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:55.800357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-9e9a2c45-bf29-454c-925e-11234419bc74 req-2fd89610-aa7b-4ed3-82f1-a3037aeb3681 service nova] Creating event network-changed:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:07:55.805846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-9e9a2c45-bf29-454c-925e-11234419bc74 req-2fd89610-aa7b-4ed3-82f1-a3037aeb3681 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048326 May 19 12:07:55.806627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 165/336] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:07:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:07:55.919298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:55.927308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:55.928828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:55.930035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ec49cb1-a7f0-4501-aeac-b2ff72a0522c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.172180 May 19 12:07:55.930594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 172/337] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:56.022024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:56.022707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:56.029414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.029852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:56.030860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.031291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:56.171867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:56.179780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-672f9aaf-423c-473e-8b88-aa4a65dcf336 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.164137 May 19 12:07:56.180356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 166/338] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:56.760845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:56.761349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:56.768544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.769084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:56.769865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.770307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:56.915428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:56.923376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cb1b5056-355b-4322-a3e0-11fce385f16a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.168949 May 19 12:07:56.923946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 173/339] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:56 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:56.949482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:56.950156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:56.956748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.957188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:56.957936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:56.958357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:57.103144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:57.113374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:57.115046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:57.116198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-af92eb11-9112-4b63-b594-0c24ee0904f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.170824 May 19 12:07:57.116759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 167/340] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:57.204128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:57.204763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:57.212488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:57.213143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:57.214265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:57.214695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:57.378463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:57.386810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dae6626a-73ac-4cd5-b8f2-1cbc35e143bc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.188725 May 19 12:07:57.387388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 174/341] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:57.940567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:57.941109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:57.947440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:57.947882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:57.948663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:57.949145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.085082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:58.093305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-88cb2318-22b0-4885-b8ea-a4677e0beb34 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.156749 May 19 12:07:58.093879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 168/342] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:57 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:58.134623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:58.135211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:58.142216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.142673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.143440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.143867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.284191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:58.293126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:58.294720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:58.296152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c384cd21-fa4c-4b09-b0ac-bb1d7989fdb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.165718 May 19 12:07:58.296972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 175/343] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:07:58.403513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:58.404120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:58.410699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.411141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.412073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.412348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.563283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:58.570797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-91d77384-6797-4ded-a33e-41551587e1a5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.171172 May 19 12:07:58.571366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 169/344] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:58.679311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2725b301-fe5f-4850-b68f-36fc1c61309f", "device": "/dev/sdb"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:58.681007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:58.689270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.689709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:58.690521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:58.691075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.111498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:59.112110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:59.119053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.119499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.120269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.120827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.270712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.278265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba25483a-5fd1-4d97-b466-54a451e70d59 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.170706 May 19 12:07:59.278835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 170/345] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:59 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:59.315254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:59.315834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:59.322142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.322586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.323392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.323814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.459801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.468048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.469576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.470734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cac7e65-7961-41d4-81d6-d801a665ea56 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.159397 May 19 12:07:59.471295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 171/346] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:07:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:07:59.475680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d8183d52-ab5f-4e48-a798-71a4aeb4ef26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:07:59.481345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8183d52-ab5f-4e48-a798-71a4aeb4ef26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.481786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8183d52-ab5f-4e48-a798-71a4aeb4ef26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.549087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d8183d52-ab5f-4e48-a798-71a4aeb4ef26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Going to try to stop instance {{(pid=117532) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 19 12:07:59.611003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d8183d52-ab5f-4e48-a798-71a4aeb4ef26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action" status: 202 len: 0 microversion: 2.1 time: 0.138137 May 19 12:07:59.611822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 172/347] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:07:59 2022] POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action => generated 0 bytes in 140 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:07:59.616997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:59.617699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:59.624456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.624951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.625705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.626131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.769908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.781539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-352963e4-025a-46d1-a12c-643bdbbdfb17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.168268 May 19 12:07:59.782254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 173/348] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:07:59.786635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:07:59.787379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:07:59.795837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.796272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.797078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:07:59.797505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:07:59.942923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:07:59.952188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4a327944-314c-4c2e-b88e-d03104b173aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.168897 May 19 12:07:59.952921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 174/349] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:07:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:00.295885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:00.296741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:00.305559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.306090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.307011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.307544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.458633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:00.469439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d4d24ed8-8adc-46d5-a095-0ce0e4694e0e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.178493 May 19 12:08:00.480288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 175/350] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:00 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:00.487374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f1dcaff2-c9d9-4526-acdb-a44068f00d85 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "POST /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.811629 May 19 12:08:00.489216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:00.489374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 176/351] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:07:58 2022] POST /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4/os-volume_attachments => generated 194 bytes in 1815 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:00.490011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:00.496209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.496924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.497903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.498532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.505090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:00.646742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:00.658253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:00.659746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:00.660947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a94a2a34-2d05-48a6-98ba-00828210cd87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.175949 May 19 12:08:00.661495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 176/352] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:00.799637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:00.800390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:00.807743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.808158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.808984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.809412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.812100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:00.949127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:00.960259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-01371fef-e5bd-4203-a585-df7d11ff7e66 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.164650 May 19 12:08:00.960994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 177/353] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:00.970746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:00.971378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:00.977576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.978018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:00.978794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:00.979219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.110939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:01.121328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1ba1caff-6939-4c82-a68d-aade4dd1f93f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.154608 May 19 12:08:01.121935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 177/354] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:01.487415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:01.488587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:01.495505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.496184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.496986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.497555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.643812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:01.652378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a3b9d9c1-744d-45f9-a6fc-64b9e45bc569 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.168730 May 19 12:08:01.652904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 178/355] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:01 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:01.679945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:01.680514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:01.686979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.687413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.688171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.688588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.823000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:01.832752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:01.834276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:01.835436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-292c95e1-0715-427c-8a7d-0429d9eb15d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.160108 May 19 12:08:01.836001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 178/356] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:01.977471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:01.978102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:01.985371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.985860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:01.986669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:01.987155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.118574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:02.126359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f49584d8-3183-4116-993d-5b68e77c1451 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.152780 May 19 12:08:02.126937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 179/357] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:08:02.139596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:02.140192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:02.146792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.147262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.148003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.148425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.289279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:02.299371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58e6ab3c-b5e8-47a5-9f48-b7dd07147d96 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.163608 May 19 12:08:02.300026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 179/358] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:02.669122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:02.669906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:02.671849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:02.672511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=18 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:02.679146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.679587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.680377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.680832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.818183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:02.826553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-30cc644b-6d39-4456-9312-f79dd3ce56d2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.161529 May 19 12:08:02.827497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 180/359] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:02 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:02.853692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:02.854399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:02.856620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:02.862057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.862678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:02.863402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:02.863898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.143707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:03.144236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:03.151392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.151942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.153008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.153471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.230705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:03.239101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:03.240594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:03.241843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c98b9a3d-3648-4484-870d-1d5e2ce8ca04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.392162 May 19 12:08:03.242698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 180/360] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:03.299779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:03.308206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-025b3b4c-6aa2-4a97-83b2-49f7bdad8906 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.168463 May 19 12:08:03.308775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 181/361] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:03.317777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:03.318384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:03.324672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.325140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.325889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.326318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.461115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:03.473862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ac8be70-bfd4-4b35-8a87-827878dc2f87 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.159599 May 19 12:08:03.474678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 181/362] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:03.847530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:03.848611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:03.859064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.859798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:03.861369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:03.862092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.000048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.009080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-98dab39f-6a07-475f-9a24-4555b3cb9820 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.168035 May 19 12:08:04.009628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 182/363] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:03 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:08:04.265159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:04.265706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:04.272166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.272602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.273418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.273823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.326497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:04.327425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:04.331622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:04.336950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.337544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.338445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.338882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.408767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.416688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.418265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.419330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-56851b3a-6045-480a-87b2-133a9bcc5a7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.162426 May 19 12:08:04.419891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 182/364] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:04.493878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:04.494590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:04.501821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.502278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.503061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:04.503483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:04.644026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.652733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7679937c-609b-4921-ad7d-6b6a5097730e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.162580 May 19 12:08:04.660731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 183/365] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:08:04.703466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:04.715347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5d59c9a-38e3-4b78-9e77-5d51ebc83cb4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.393373 May 19 12:08:04.715932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 183/366] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:08:05.028397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:05.029185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:05.037720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.038326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.039347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.039947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.175618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.187930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c644f632-9b6b-47c4-8e70-e728d506727c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.164200 May 19 12:08:05.188679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 184/367] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:05 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:05.439095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:05.439656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:05.446686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.447133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.447887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.448310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.506458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-6271e535-1559-459e-bbef-f49d42c685f7 req-6937ce15-cd1b-4b3b-b88e-9a2ab465483b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7b377f1-b40c-4146-8569-44510a0c33ab", "name": "network-vif-plugged", "status": "completed", "tag": "1bd50345-4513-45c1-ae8b-577dff600ff2"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:05.517645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-6271e535-1559-459e-bbef-f49d42c685f7 req-6937ce15-cd1b-4b3b-b88e-9a2ab465483b service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.518295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-6271e535-1559-459e-bbef-f49d42c685f7 req-6937ce15-cd1b-4b3b-b88e-9a2ab465483b service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.533756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-6271e535-1559-459e-bbef-f49d42c685f7 req-6937ce15-cd1b-4b3b-b88e-9a2ab465483b service nova] Creating event network-vif-plugged:1bd50345-4513-45c1-ae8b-577dff600ff2 for instance a7b377f1-b40c-4146-8569-44510a0c33ab on hv-cinder-80403 May 19 12:08:05.539587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-6271e535-1559-459e-bbef-f49d42c685f7 req-6937ce15-cd1b-4b3b-b88e-9a2ab465483b service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044634 May 19 12:08:05.540735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 185/368] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:05.581803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.589900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.591398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.592561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d828438f-c5f9-41e0-83cb-d5b3ed930388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.157616 May 19 12:08:05.593342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 184/369] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:05.670882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:05.671566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:05.677933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.678433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.679189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.679615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.732094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:05.733050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:05.752069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.752537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.753333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:05.753756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:05.842828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.851462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d57e3c5d-a27e-42d3-98c0-076f3a278cfe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.184816 May 19 12:08:05.852038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 186/370] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:05.899108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:05.906801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7806b0ad-fe62-4c3c-9329-dd80fe5a5c5c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.178805 May 19 12:08:05.907408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 185/371] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:06.208750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:06.209542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:06.217933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.218362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.219130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.219555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.356934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:06.364486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c3fe3009-e90f-4f8d-b706-ca7a3eba0fe8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.161890 May 19 12:08:06.365076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 187/372] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:06 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:06.615369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:06.616366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:06.627394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.628116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.629441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.630143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.768890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:06.777481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:06.778965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:06.780130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e055db40-49d1-4c79-bdc4-14dabfc27ada tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.171336 May 19 12:08:06.780701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 186/373] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:06.869568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:06.870378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:06.876550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.877028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.877811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.878239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.924700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:06.925536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:06.932276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.932719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:06.933721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:06.934311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.025897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.038001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-26fa68af-f6a1-4bfd-8e62-3a96178774c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.172381 May 19 12:08:07.038596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 188/374] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:07.092169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.094592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1522c50f-3419-48a9-9292-2246acef31a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.174472 May 19 12:08:07.095241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 187/375] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:07.383988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:07.384678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:07.391702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:07.392140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.392920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:07.393344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.536870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.546444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b4bdec6a-b222-4be3-8119-b0f994e3d915 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.168979 May 19 12:08:07.546995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 189/376] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:07 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:07.555479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-9db91fb6-93ae-49a9-9f0b-07f7d1dbdf93 req-56e948e8-ccb2-4f17-b98f-72efcb2a6ecc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "name": "network-vif-plugged", "status": "completed", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:07.564229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9db91fb6-93ae-49a9-9f0b-07f7d1dbdf93 req-56e948e8-ccb2-4f17-b98f-72efcb2a6ecc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:07.564667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9db91fb6-93ae-49a9-9f0b-07f7d1dbdf93 req-56e948e8-ccb2-4f17-b98f-72efcb2a6ecc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.575847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-9db91fb6-93ae-49a9-9f0b-07f7d1dbdf93 req-56e948e8-ccb2-4f17-b98f-72efcb2a6ecc service nova] Creating event network-vif-plugged:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:08:07.580829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-9db91fb6-93ae-49a9-9f0b-07f7d1dbdf93 req-56e948e8-ccb2-4f17-b98f-72efcb2a6ecc service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028896 May 19 12:08:07.581393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 188/377] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:07.800803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:07.801496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:07.809636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:07.810078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.810819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:07.811243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:07.950290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.958768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.960256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:07.961419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-10b212e9-eb93-41bb-9856-539f2e7429c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.165985 May 19 12:08:07.961974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 190/378] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:08.058840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:08.059502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:08.068985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.069497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.070507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.071014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.111548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:08.112159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:08.119758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.120206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.121134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.121566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.211439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:08.220005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-95f08664-ccad-4c8d-a43e-0e6eacdb1b44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.167041 May 19 12:08:08.220350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 189/379] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:08.260853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:08.269768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d2da8713-3a3b-4b68-b6e7-66ad8ae223f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.161779 May 19 12:08:08.270813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 191/380] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:08.563571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:08.564200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:08.570723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.571160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.571902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.572323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.717035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:08.728906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-85f603b2-d333-4ccb-95d4-7d419e538866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.169169 May 19 12:08:08.729819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 190/381] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:08 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:08.984312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:08.985316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:08.995807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.996541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:08.997829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:08.998528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.134149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.147052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.149058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.150803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f9ab69c0-b477-48bd-bddb-c77bb9057b10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.172966 May 19 12:08:09.151658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 192/382] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:09.236611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:09.237158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:09.244023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.244471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.245292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.245699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.289702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:09.290284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:09.297413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.297849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.298618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.299109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.318266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:09.396129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.405032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a310363-2e03-41d5-811c-8895fad06bea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.172190 May 19 12:08:09.405833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 191/383] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:09.444278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.453821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d505ce5c-b3fc-4a16-ab1b-073ed62cdb76 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.167675 May 19 12:08:09.454417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 193/384] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:09.750195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:09.751231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:09.761833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.762275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.763027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:09.763450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:09.895645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:09.903269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c811f471-8a06-4793-a9ce-a51f21ba2a74 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.159570 May 19 12:08:09.903836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 192/385] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:09 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:10.169810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:10.170410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:10.177021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.177469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.178253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.178676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.313303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:10.321496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:10.322983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:10.324186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1c801034-bee7-45b1-8724-2f08fe032d34 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.158340 May 19 12:08:10.324785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 194/386] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:10.428229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:10.429338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:10.440146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.440944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.442192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.442907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.470524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:10.471129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:10.479459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.479951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.480735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.481170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.579175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:10.587273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7bceec7f-ea94-4392-b2de-93cff06d07cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.165687 May 19 12:08:10.587852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 193/387] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:10.620396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:10.628137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-999d753c-f9c7-41b6-943b-1e83d2cb57f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.161415 May 19 12:08:10.628747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 195/388] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:10.920144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:10.920770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:10.927211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.927647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.928398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:10.928876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:10.943925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:11.065779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.074450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-52af10c7-42d4-4a67-b132-a2f66374c0df tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1266 microversion: 2.1 time: 0.158295 May 19 12:08:11.075023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 194/389] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:10 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:11.342652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:11.343221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:11.349894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.350317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.351089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.351492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.484710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.496621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.498159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.499290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4d8f4e23-6542-42b6-98d1-1ff1084b7344 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.160743 May 19 12:08:11.499849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 196/390] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:11.608806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:11.609748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:11.620806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.621253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.622013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.622462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.645569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:11.646172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:11.652592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.653072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.653834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:11.654260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:11.762527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.771019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bb1b4b6d-d4ad-41bb-8a4b-9048c3146172 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.168787 May 19 12:08:11.771606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 195/391] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:11.785493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:11.793101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e9c3c3b4-7988-4a6c-91d5-283c72063763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.151365 May 19 12:08:11.793713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 197/392] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:12.091573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:12.092237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:12.098571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.099017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.099774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.100201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.240202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.247844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c4e31f45-5ddb-46ec-b515-27011a35d784 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1454 microversion: 2.1 time: 0.160481 May 19 12:08:12.248410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 196/393] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:12 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1454 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:12.265197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:12.265800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:12.271974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.272501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.274040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.274476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.414760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.422987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b0f07ada-1ffb-4163-b1ef-44c49cffeabb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1454 microversion: 2.1 time: 0.162152 May 19 12:08:12.423562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 198/394] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:12 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1454 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:12.518301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:12.518906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:12.526069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.526535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.527280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.527699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.661560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.669693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.671167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.672316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bb97116c-f450-462e-bcd4-aeedced1f1b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.158091 May 19 12:08:12.672919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 197/395] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:12.790695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:12.791307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:12.800315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.800824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.801649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.802086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.815111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:12.816014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:12.823034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.824042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.824887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:12.825313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:12.957028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:12.964724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9bab25ac-2afe-4a29-a46d-d0c7f8bb447e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.153459 May 19 12:08:12.965302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 198/396] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:13.019328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:13.027915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2bf2d9aa-d3b4-45ce-9b18-a062e54d96a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.241519 May 19 12:08:13.028667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 199/397] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:13.691938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:13.692781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:13.701423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:13.702038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:13.703519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:13.704008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:13.864819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:13.876458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:13.878410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:13.879592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-02d25a81-b8df-4ab2-b625-2223f526eaeb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1721 microversion: 2.20 time: 0.192492 May 19 12:08:13.880165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 199/398] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1721 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:13.982894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:13.983608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:13.990093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:13.990560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:13.991356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:13.991786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:14.046745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:14.047203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:14.053662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:14.054102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:14.054863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:14.055291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:14.189145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:14.197164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f8ff6367-fee8-49ad-853c-88c00ed951d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.154460 May 19 12:08:14.197740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 200/399] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:14.213542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:14.221481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-af6ffac8-9467-446e-b82c-ee22b5687c0f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1372 microversion: 2.1 time: 0.242929 May 19 12:08:14.222282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 200/400] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1372 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:14.457738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-d639a7b5-74a7-47d0-a86d-f5e0f40e6474 req-55908795-e88b-445d-9bf1-1f072589d317 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7b377f1-b40c-4146-8569-44510a0c33ab", "tag": "1bd50345-4513-45c1-ae8b-577dff600ff2"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:14.466721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-d639a7b5-74a7-47d0-a86d-f5e0f40e6474 req-55908795-e88b-445d-9bf1-1f072589d317 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:14.467488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-d639a7b5-74a7-47d0-a86d-f5e0f40e6474 req-55908795-e88b-445d-9bf1-1f072589d317 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:14.479852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-d639a7b5-74a7-47d0-a86d-f5e0f40e6474 req-55908795-e88b-445d-9bf1-1f072589d317 service nova] Creating event network-changed:1bd50345-4513-45c1-ae8b-577dff600ff2 for instance a7b377f1-b40c-4146-8569-44510a0c33ab on hv-cinder-80403 May 19 12:08:14.484840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-d639a7b5-74a7-47d0-a86d-f5e0f40e6474 req-55908795-e88b-445d-9bf1-1f072589d317 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030791 May 19 12:08:14.485555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 201/401] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:14.898310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:14.899574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:14.905790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:14.906196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:14.907112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:14.907690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.055448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:15.064933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:15.066679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:15.068170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-acd90b68-b221-44f8-a14f-d2a060e05cfd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.173666 May 19 12:08:15.068898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 201/402] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:15.216011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:15.216766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:15.223481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.223683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.225141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.225687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.240812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:15.241862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:15.250386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.251077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.251870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.252332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.366236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:15.375223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86cd345c-e87d-4274-a2e6-ae1cbecb40f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.164107 May 19 12:08:15.377020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 202/403] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:15.397111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:15.405405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-52de68ee-b6f9-4235-969b-f9eb65e82bbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1566 microversion: 2.1 time: 0.168676 May 19 12:08:15.405979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 202/404] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1566 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:15.422140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:15.422628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:15.429074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.429589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.430417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:15.430928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:15.516255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11a5b1e2-997b-4825-9412-9eacac9aa0d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098167 May 19 12:08:15.516980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 203/405] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:08:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:08:16.087692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:16.088303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:16.094751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:16.095186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:16.095939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:16.096357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:16.233342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:16.241424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:16.242905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:16.244064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4ea46bdb-45d2-442a-ac5f-de67dfc1478b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.160340 May 19 12:08:16.244609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 203/406] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:16.394996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:16.395606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:16.402660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:16.403102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:16.403862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:16.404289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:16.545783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-e6d5f6ea-0c9e-4ba0-9521-c133bd4218a2 req-08cd2b1a-2e5b-419f-994b-649108ad4639 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:16.553771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6d5f6ea-0c9e-4ba0-9521-c133bd4218a2 req-08cd2b1a-2e5b-419f-994b-649108ad4639 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:16.554191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6d5f6ea-0c9e-4ba0-9521-c133bd4218a2 req-08cd2b1a-2e5b-419f-994b-649108ad4639 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:16.565718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-e6d5f6ea-0c9e-4ba0-9521-c133bd4218a2 req-08cd2b1a-2e5b-419f-994b-649108ad4639 service nova] Creating event network-changed:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:08:16.571337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-e6d5f6ea-0c9e-4ba0-9521-c133bd4218a2 req-08cd2b1a-2e5b-419f-994b-649108ad4639 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028882 May 19 12:08:16.571902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 204/407] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:08:16.573899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:16.582625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e1583af2-f61d-43ea-af8c-7f190a9a1bb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.192698 May 19 12:08:16.583311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 204/408] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:17.263428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:17.264166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:17.273410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:17.273410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:17.273410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:17.273410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:17.404175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:17.412919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:17.414453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:17.415634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ca564a4d-1376-440c-9c6a-5ec83536514c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.156626 May 19 12:08:17.416167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 205/409] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:17.600402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:17.601036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:17.609719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:17.610211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:17.611181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:17.611838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:17.744170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:17.752295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c129650f-ff8c-4ec3-99bf-7a14262c8aeb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.155852 May 19 12:08:17.752891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 205/410] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:18.223490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-07014c93-a0cf-4dc9-a0b8-194575bf9c86 req-db877a72-0d8e-468e-acef-8265c17d15d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "name": "network-vif-plugged", "status": "completed", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:18.232656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-07014c93-a0cf-4dc9-a0b8-194575bf9c86 req-db877a72-0d8e-468e-acef-8265c17d15d2 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:18.233143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-07014c93-a0cf-4dc9-a0b8-194575bf9c86 req-db877a72-0d8e-468e-acef-8265c17d15d2 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:18.245052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-07014c93-a0cf-4dc9-a0b8-194575bf9c86 req-db877a72-0d8e-468e-acef-8265c17d15d2 service nova] Creating event network-vif-plugged:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:08:18.249951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-07014c93-a0cf-4dc9-a0b8-194575bf9c86 req-db877a72-0d8e-468e-acef-8265c17d15d2 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030344 May 19 12:08:18.250542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 206/411] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:18.434828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:18.435424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:18.441731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:18.442168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:18.442912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:18.443333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:18.585577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:18.594315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:18.595921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:18.597159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f976e67-f4ee-4c7a-aff0-f649c3036492 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.166632 May 19 12:08:18.597724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 206/412] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:18.770897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:18.771729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:18.775092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:18.780011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:18.781164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:18.781930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:18.782377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:18.922866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:18.930897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cefca1cb-0ec9-4822-953b-d0a7cc7fbab9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.164242 May 19 12:08:18.931469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 207/413] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:19.619479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:19.620221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:19.627172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:19.627609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:19.628352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:19.628809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:19.782853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:19.795969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:19.798099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:19.799827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e9502229-06a4-4939-a4cb-fabc127a1ace tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.186761 May 19 12:08:19.800539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 207/414] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:19.948894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:19.949500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:19.955966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:19.956415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:19.957209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:19.957636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:20.104455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:20.112758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e4e2a8f-d01e-4590-af8b-69e3b73e656a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.167860 May 19 12:08:20.113333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 208/415] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:20.825107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:20.825800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:20.833877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:20.834384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:20.835251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:20.835746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:20.980020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:20.988362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:20.989865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:20.991019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3925de83-3d74-4e0f-af2b-f3bf67f0eb24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.172589 May 19 12:08:20.991586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 208/416] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:21.134250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:21.135267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:21.145775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:21.146480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:21.147706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:21.148398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:21.285224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:21.293438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b0a9666b-c008-49e7-b31b-5574c3262486 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.166110 May 19 12:08:21.294019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 209/417] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:22.014187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:22.015133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:22.025762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:22.026447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:22.027715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:22.028451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:22.169515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:22.181731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:22.183800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:22.184992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dbc28183-82cc-43bb-9234-ecd2f7ca2317 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.178050 May 19 12:08:22.185718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 209/418] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:22.314374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:22.315379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:22.325411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:22.325851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:22.326605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:22.327025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:22.468272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:22.476842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a8f9132e-2e5a-45f2-b6c8-697f0cf00d38 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.168824 May 19 12:08:22.477411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 210/419] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:08:23.204076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:23.204632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:23.211716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:23.212150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:23.213008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:23.213425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:23.359397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:23.367529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:23.369055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:23.370222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0d07ad30-e811-468c-b0bc-24c6b55930f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.170336 May 19 12:08:23.370792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 210/420] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:23.498100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:23.498651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:23.505508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:23.505977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:23.507124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:23.507570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:23.526976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:23.657976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:23.670596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-17591d44-ae1d-4945-98bf-686eeb3f3c20 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.178603 May 19 12:08:23.671162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 211/421] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:24.391439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:24.392151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:24.399947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:24.400439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:24.401330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:24.401780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:24.538773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:24.547033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:24.548509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:24.549684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-411a5543-df97-4161-95f9-7a44f8c6744b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.163146 May 19 12:08:24.550244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 211/422] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:24.691689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:24.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:24.698913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:24.699354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:24.700170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:24.700760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:24.831349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:24.839602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f690a49b-d826-4551-805c-69a98906abe3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.151890 May 19 12:08:24.840262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 212/423] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:25.571423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:25.572461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:25.583710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:25.584434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:25.585714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:25.586458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:25.593162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:25.719557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:25.727466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:25.728981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:25.730143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-95799fbd-a7c1-44d0-bd50-3f1c4c8479d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1742 microversion: 2.20 time: 0.165386 May 19 12:08:25.730742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 212/424] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1742 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:25.860959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:25.861875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:25.872031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:25.872469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:25.873364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:25.873786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:26.062216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:26.070403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-278e4d77-77b5-458f-8b95-8b28a0b91378 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.215920 May 19 12:08:26.070996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 213/425] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:26.754957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:26.756977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:26.764232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:26.765033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:26.766025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:26.766678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:26.988862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:26.998655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:27.000094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:27.001695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-17e0cd7d-575b-49ca-bdbc-8db66b8ea9bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.251240 May 19 12:08:27.002517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 213/426] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:27.087970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:27.088656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:27.104435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:27.104916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:27.105697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:27.106173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:27.325302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:27.336531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c5ece176-1e57-4b65-bc7d-458d9922c182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.252383 May 19 12:08:27.337852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 214/427] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:28.021736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:28.022413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:28.031494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:28.032380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:28.033104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:28.033690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:28.183532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:28.192462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:28.193997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:28.195437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-004ba2ca-50b5-4d9a-886b-ea49a7f2d87c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.177797 May 19 12:08:28.196209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 214/428] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:28.357383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:28.359185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:28.366119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:28.367217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:28.368447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:28.369483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:28.542243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:28.552045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a0356f05-9c41-4ef3-8899-941e79fa98e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.199292 May 19 12:08:28.556670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 215/429] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:29.217382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:29.218026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:29.224751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:29.225144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:29.225901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:29.226325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:29.377993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:29.394575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:29.397363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:29.399222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3c688d00-206b-4a40-bd4d-0277916bea7e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.187536 May 19 12:08:29.399795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 215/430] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:29.569815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:29.570397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:29.576993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:29.577473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:29.578247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:29.578858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:29.727595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:29.745320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b360041f-a085-4b6b-b345-efdf565ffaf1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.179489 May 19 12:08:29.745905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 216/431] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:30.422863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:30.423453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:30.431626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:30.432071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:30.432893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:30.433335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:30.609109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:30.618433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:30.620332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:30.621744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e9052500-a265-436e-b53b-c6b0f86425d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.204888 May 19 12:08:30.622327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 216/432] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:30.765280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:30.766507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:30.773152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:30.773855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:30.774822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:30.776116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:30.942654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:30.952320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b6fd0b44-58d4-4339-94d3-6d2e16a82be9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1722 microversion: 2.1 time: 0.191316 May 19 12:08:30.970927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 217/433] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1722 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:31.645341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:31.646857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:31.654990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:31.655891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:31.657312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:31.657944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:31.804926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:31.813487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:31.815055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:31.816233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b93c4f85-c185-44df-9293-102a3f32a479 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.177696 May 19 12:08:31.827464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 217/434] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:31.970117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:31.971878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:31.978973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:31.979803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:31.980783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:31.981394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:32.153200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:32.165237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-026644aa-0266-4cbc-9538-88f620efdef3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1699 microversion: 2.1 time: 0.199186 May 19 12:08:32.165933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 218/435] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1699 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:32.183762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eaf201a2-86da-46a1-a5e6-cdafd5ca6de4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:32.191634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaf201a2-86da-46a1-a5e6-cdafd5ca6de4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:32.192189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaf201a2-86da-46a1-a5e6-cdafd5ca6de4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:32.273845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eaf201a2-86da-46a1-a5e6-cdafd5ca6de4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Going to try to start instance {{(pid=117533) start /opt/stack/nova/nova/compute/api.py:2730}} May 19 12:08:32.340221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eaf201a2-86da-46a1-a5e6-cdafd5ca6de4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action" status: 202 len: 0 microversion: 2.1 time: 0.151556 May 19 12:08:32.340221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 218/436] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:08:32 2022] POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action => generated 0 bytes in 156 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:08:32.340740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:32.340740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:32.347389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:32.347914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:32.349053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:32.349638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:32.503883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:32.516838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-97653cd1-ee1e-43bc-9b5f-d5190380b206 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1708 microversion: 2.1 time: 0.181228 May 19 12:08:32.517430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 219/437] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1708 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:32.836075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:32.841744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:32.850567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:32.851159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:32.853052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:32.853815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:33.130568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:33.139538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:33.141136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:33.142516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d00ae008-cef2-4881-bfdd-fb4d1bae52ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.311564 May 19 12:08:33.155738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 219/438] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:33.534548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:33.535801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:33.542546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:33.543252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:33.544023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:33.544507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:33.690820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:33.699062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-53afcd82-92ba-4b64-9028-6e96b64b136a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1708 microversion: 2.1 time: 0.168633 May 19 12:08:33.699673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 220/439] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1708 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:34.162310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:34.163634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:34.171355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.171790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:34.172553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.173006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:34.327795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:34.337304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:34.338787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:34.339980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f056ee80-3b71-4eda-8c63-cf573c07db33 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.181643 May 19 12:08:34.340725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 220/440] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:34.717146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:34.718602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:34.725307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.725891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:34.726789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.727319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:34.892169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:34.900987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e4a82f73-c6b5-445e-90fd-c4af26f6a0ff tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1712 microversion: 2.1 time: 0.188390 May 19 12:08:34.912866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 221/441] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1712 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:34.969086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:34.969972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:34.972061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=75 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:34.977667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.978497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:34.979420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:34.980141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:35.118636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:08:35.281483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f3339a73-5d57-4935-9ed4-3d29b5b47f9f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4/os-volume_attachments/2725b301-fe5f-4850-b68f-36fc1c61309f" status: 202 len: 0 microversion: 2.1 time: 0.317429 May 19 12:08:35.283256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 221/442] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:08:34 2022] DELETE /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4/os-volume_attachments/2725b301-fe5f-4850-b68f-36fc1c61309f => generated 0 bytes in 320 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:08:35.359302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:35.360041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:35.366493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:35.367269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:35.368041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:35.368731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:35.533006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:35.541184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:35.542659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:35.543817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96f9b26f-619c-4ab6-8ddb-7ce9fb32a0e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.188662 May 19 12:08:35.544458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 222/443] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:36.563263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:36.564445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:36.570373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:36.570812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:36.571550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:36.571966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:36.713961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:36.722294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:36.723773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:36.724981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-494738c0-6d44-4087-8ad5-7de71f141664 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1737 microversion: 2.20 time: 0.166505 May 19 12:08:36.725572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 222/444] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:37.481438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:37.481438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:37.487196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.487561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.488412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.488835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.522078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:08:37.610085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-48ea1bbe-a4b9-41b1-ab57-148de85e1045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 204 len: 0 microversion: 2.1 time: 0.133172 May 19 12:08:37.610820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 223/445] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:08:37 2022] DELETE /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:08:37.619297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:37.619925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:37.626198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.626634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.627389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.627814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.743243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:37.744086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:37.750352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.750800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.751579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:37.752008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:37.764371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:37.772675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-43aa2941-652b-47ab-a69c-092e47a5bc92 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1512 microversion: 2.1 time: 0.157350 May 19 12:08:37.773549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 223/446] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:37 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1512 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:37.897743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:37.906649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:37.908161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:37.909422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01254d49-f18f-48e4-9545-70d42edf6a60 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1719 microversion: 2.20 time: 0.170689 May 19 12:08:37.909971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 224/447] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1719 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:38.790423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:38.790950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:38.798076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:38.798650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:38.799287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:38.799699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:38.927699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:38.928996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:38.935284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:38.935659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:38.936123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:38.937080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:38.937662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:38.945544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b86d696a-f3e6-4b5c-9efd-cd8e24e2ed42 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1512 microversion: 2.1 time: 0.158873 May 19 12:08:38.946129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 224/448] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:38 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1512 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:39.104408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:39.112883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:39.114374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:39.115539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e8cee74-e0bd-41a8-96b5-b7f6bdb01f15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1719 microversion: 2.20 time: 0.191826 May 19 12:08:39.116103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 225/449] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1719 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:08:39.967286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:39.967910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:39.976061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:39.976603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:39.977498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:39.978001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:40.114683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:40.122635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d45521fd-82fa-4591-98e4-1ade0c050ecc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1512 microversion: 2.1 time: 0.161547 May 19 12:08:40.123380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 225/450] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:39 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1512 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:40.133933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:40.134560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:40.140884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:40.141320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:40.142043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:40.142465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:40.160350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:40.281856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:40.289998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:40.291464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:40.292745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f0a26d84-0548-40cc-86eb-d8b3983bf8fb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1719 microversion: 2.20 time: 0.163055 May 19 12:08:40.293356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 226/451] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1719 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:40.571041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-928c78e8-5a38-4780-bd1c-5e1bfffc9834 req-260afa75-b4d6-41d6-89b3-05c0091d547a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "182ed917-1ffe-42f8-9af3-65bcb94ae5c4", "name": "network-vif-unplugged", "status": "completed", "tag": "f38c2817-0741-4776-a66b-c6da3da4ed42"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:40.581370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-928c78e8-5a38-4780-bd1c-5e1bfffc9834 req-260afa75-b4d6-41d6-89b3-05c0091d547a service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:40.581827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-928c78e8-5a38-4780-bd1c-5e1bfffc9834 req-260afa75-b4d6-41d6-89b3-05c0091d547a service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:40.594224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-928c78e8-5a38-4780-bd1c-5e1bfffc9834 req-260afa75-b4d6-41d6-89b3-05c0091d547a service nova] Creating event network-vif-unplugged:f38c2817-0741-4776-a66b-c6da3da4ed42 for instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 on hv-cinder-80403 May 19 12:08:40.599563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-928c78e8-5a38-4780-bd1c-5e1bfffc9834 req-260afa75-b4d6-41d6-89b3-05c0091d547a service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032310 May 19 12:08:40.600261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 226/452] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:40 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:41.146117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:41.147141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:41.157567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:41.158358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:41.159601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:41.160358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:41.313913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:41.316179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:41.317266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:41.323779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:41.323914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7ec27d71-9602-476e-9584-b45b113a6e05 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1512 microversion: 2.1 time: 0.184312 May 19 12:08:41.324223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:41.325101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:41.325524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:41.329098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 227/453] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:41 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1512 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:41.466322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:41.468670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:41.475793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:41.477301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:41.478458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c2145578-26a6-4f36-a8f5-865f2df120ab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1719 microversion: 2.20 time: 0.169806 May 19 12:08:41.479009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 227/454] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1719 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:42.341932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:42.342578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:42.348703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.349148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.349898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.350320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.481273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:42.496758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bd1d0209-34f0-4c55-8427-90d7a02f5664 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1512 microversion: 2.1 time: 0.159243 May 19 12:08:42.498624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:42.499382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:42.508008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.508691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.509565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.509992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.510338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 228/455] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:42 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1512 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:42.623959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-e6ce45b3-9c4e-4155-839a-de3b4df64580 req-24f3ae52-8550-4a44-8729-1c334fbe9727 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "name": "network-vif-unplugged", "status": "completed", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:42.633347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6ce45b3-9c4e-4155-839a-de3b4df64580 req-24f3ae52-8550-4a44-8729-1c334fbe9727 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.633873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6ce45b3-9c4e-4155-839a-de3b4df64580 req-24f3ae52-8550-4a44-8729-1c334fbe9727 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.648235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-e6ce45b3-9c4e-4155-839a-de3b4df64580 req-24f3ae52-8550-4a44-8729-1c334fbe9727 service nova] Creating event network-vif-unplugged:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:08:42.656249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-e6ce45b3-9c4e-4155-839a-de3b4df64580 req-24f3ae52-8550-4a44-8729-1c334fbe9727 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036264 May 19 12:08:42.656904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 229/456] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:42 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:08:42.768359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-de3f5e3e-a772-4a27-a008-a9273c1e52b2 req-c7573dc7-99bd-4a0d-8165-bc99de914f73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "182ed917-1ffe-42f8-9af3-65bcb94ae5c4", "name": "network-vif-deleted", "tag": "f38c2817-0741-4776-a66b-c6da3da4ed42"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:42.787401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-de3f5e3e-a772-4a27-a008-a9273c1e52b2 req-c7573dc7-99bd-4a0d-8165-bc99de914f73 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.788325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-de3f5e3e-a772-4a27-a008-a9273c1e52b2 req-c7573dc7-99bd-4a0d-8165-bc99de914f73 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.802169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-de3f5e3e-a772-4a27-a008-a9273c1e52b2 req-c7573dc7-99bd-4a0d-8165-bc99de914f73 service nova] Creating event network-vif-deleted:f38c2817-0741-4776-a66b-c6da3da4ed42 for instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 on hv-cinder-80403 May 19 12:08:42.806481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:42.807491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-de3f5e3e-a772-4a27-a008-a9273c1e52b2 req-c7573dc7-99bd-4a0d-8165-bc99de914f73 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044152 May 19 12:08:42.808581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 230/457] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:08:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:08:42.817139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:42.819820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:42.822004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b5ffdafa-0108-44e7-95c4-9b0d8d6b1dd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1722 microversion: 2.20 time: 0.328648 May 19 12:08:42.823043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 228/458] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1722 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:08:42.841045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "657bcb07-4e79-4f43-b97d-9210968ea599"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:42.842227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:42.848889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.849318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:42.850085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:42.850507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.185089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d6af4fd2-8e79-49ce-9279-c1b5f76fe612 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.348447 May 19 12:08:43.185707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 231/459] 10.222.0.35 () {62 vars in 1363 bytes} [Thu May 19 12:08:42 2022] POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 188 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:08:43.307342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:43.308045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:43.314433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.314869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.315642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.316064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.419030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-94265fe6-9165-4cc7-b87c-1f3950f0d485 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/action" status: 202 len: 0 microversion: 2.20 time: 0.115424 May 19 12:08:43.422730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 229/460] 10.222.0.35 () {62 vars in 1318 bytes} [Thu May 19 12:08:43 2022] POST /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/action => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 19 12:08:43.428036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:43.428663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:43.438184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.438184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.439168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.439712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.518148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:43.519160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:43.526425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.526425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.528942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.529430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.571050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:43.571816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:43.572420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'fault' on Instance uuid 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:08:43.591311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:43.600585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:43.602127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:43.603551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5dc06f05-1b74-4f5d-842c-466bad1790cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1748 microversion: 2.20 time: 0.179473 May 19 12:08:43.604120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 232/461] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1748 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:43.621949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:43.629473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f18f1221-df40-4ee4-be16-19ad6427a951 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 200 len: 1307 microversion: 2.1 time: 0.116605 May 19 12:08:43.630048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 230/462] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:43 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 1307 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:44.626042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:44.626841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:44.635073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:44.635855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:44.637266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:44.637809 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:44.647235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:44.647806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: 182ed917-1ffe-42f8-9af3-65bcb94ae5c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:44.655304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:44.655841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:44.656611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:44.657348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:44.671677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 HTTP/1.1 May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:08:44.673390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. May 19 12:08:44.674942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '182ed917-1ffe-42f8-9af3-65bcb94ae5c4'} May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:08:44.675816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. May 19 12:08:44.677595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:08:44.678894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 404 to user: Instance 182ed917-1ffe-42f8-9af3-65bcb94ae5c4 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:08:44.678894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a61dc09d-fc93-4c04-bc7f-c256228e76b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4" status: 404 len: 111 microversion: 2.1 time: 0.031623 May 19 12:08:44.680610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 231/463] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:44 2022] GET /compute/v2.1/servers/182ed917-1ffe-42f8-9af3-65bcb94ae5c4 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 19 12:08:44.793407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:44.802400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:44.803546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:44.804736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6440a5b8-75db-4efc-be2c-8281cc5be69d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1748 microversion: 2.20 time: 0.185659 May 19 12:08:44.805318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 233/464] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1748 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:45.827218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:45.828175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:45.839317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:45.840057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:45.841078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:45.841496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:45.975984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:45.989713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:45.992190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:45.994047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7102e74a-8685-48b6-9555-6afa63373a0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.173073 May 19 12:08:45.994875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 232/465] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:47.017982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:47.018899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:47.029754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:47.030491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:47.031745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:47.032468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:47.175568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:47.188547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:47.190923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:47.192791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2fafd16a-7618-4409-b239-68ddfbf94967 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.181195 May 19 12:08:47.193548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 234/466] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:48.216938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:48.217625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:48.227813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:48.228443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:48.229620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:48.230230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:48.369706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:48.378185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:48.379653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:48.381179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d514f7ea-e06d-47e1-8f21-94c814531eec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.171154 May 19 12:08:48.381743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 233/467] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:49.399450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:49.400045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:49.402134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:49.407152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:49.407605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:49.408335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:49.408777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:49.545307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:49.553980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:49.555435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:49.556702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d280d129-5011-4f30-89ad-0a3494e9e353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.161249 May 19 12:08:49.557296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 235/468] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:50.579761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:50.580759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:50.591349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:50.591790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:50.592545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:50.592999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:50.725869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:50.734719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:50.736245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:50.737428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-20a8cb87-477f-4394-9cce-054a7205e0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.164638 May 19 12:08:50.737957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 234/469] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:51.756368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:51.757039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:51.763480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:51.763923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:51.764716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:51.765143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:51.899708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:51.908224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:51.909745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:51.910912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4bc24919-31a7-492e-b119-99ddf97dd84a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1750 microversion: 2.20 time: 0.158985 May 19 12:08:51.911471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 236/470] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1750 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:52.929059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:52.929665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:52.936278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:52.936747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:52.937506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:52.937930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:53.086101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:53.095404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:53.097128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:53.098495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fc3b9061-98ea-4915-a767-5574d7208252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.173497 May 19 12:08:53.099057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 235/471] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:53.517829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:08:53.518464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:08:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:08:54.116805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:54.117405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:54.124062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:54.124512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:54.125314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:54.125738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:54.273188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:54.282226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:54.283703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:54.284893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a6e3c983-7c13-4bf8-aec4-6f293fb2a2d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.171994 May 19 12:08:54.285431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 237/472] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:55.308333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:55.309590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:55.321786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:55.322512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:55.323743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:55.324783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:55.520391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:55.529854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:55.531340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:55.532507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36aa9685-d3b2-426e-8701-36dec1a5e9bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.231570 May 19 12:08:55.533103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 236/473] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:56.554916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:56.557164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:56.569674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:56.570510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:56.572833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:56.574233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:56.775520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-435629580", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "networks": [{"uuid": "598be137-c8c2-4296-bb8a-3d04b858ef0b"}]}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:08:56.825606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:56.848481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:56.848481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:56.851072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0662a0cc-c1b9-41d8-bebd-3a64091a7abd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.303171 May 19 12:08:56.856808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 238/474] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:56.873316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] validate_networks() for [('598be137-c8c2-4296-bb8a-3d04b858ef0b', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:08:57.237575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:08:57.238059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:08:57.238607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:08:57.239073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:08:57.239542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:08:57.248760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:57.249279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:57.250129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:57.250627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:57.268039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Getting quotas for project 92c49142ea324049ba24b55ff84e5a3b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:08:57.271955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Getting quotas for user d059ba56941148f3bb6080ba4f537f9d and project 92c49142ea324049ba24b55ff84e5a3b. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:08:57.283124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:08:57.285431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:08:57.331027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-927dbbc4-db6c-47d1-8740-5dfd0c26d9f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.559761 May 19 12:08:57.331685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 237/475] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:08:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 561 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:08:57.341676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-94703cc4-5d69-44fb-85ed-af20fdbfa28b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:57.342283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-94703cc4-5d69-44fb-85ed-af20fdbfa28b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:57.422266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-94703cc4-5d69-44fb-85ed-af20fdbfa28b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:57.429299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-94703cc4-5d69-44fb-85ed-af20fdbfa28b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1218 microversion: 2.1 time: 0.091541 May 19 12:08:57.440866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 239/476] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1218 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:57.870256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:57.871033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:57.878046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:57.878772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:57.880274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:57.880813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:58.021673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:58.030501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:58.031970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:58.033162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3441d213-f6d0-4a53-b7e6-6becdbaf7702 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.167532 May 19 12:08:58.033721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 238/477] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:08:58.452163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:58.453490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:58.460476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:58.460943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:58.461797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:58.462397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:58.528135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:58.536570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0521fb5c-28eb-49ca-afbb-e87aeda0c344 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.088305 May 19 12:08:58.537158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 240/478] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:08:59.051284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:59.051890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:59.058778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:59.059219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:59.060127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:59.060490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:59.225704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:59.243126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:59.244605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:59.245984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-39524c9d-5039-4c97-8bdd-3a9f3edd12f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.198517 May 19 12:08:59.246684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 239/479] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:08:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:08:59.553432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:08:59.554356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:08:59.561010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:59.561636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:59.562335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:08:59.562850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:08:59.690021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:08:59.700113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-be72c37b-f649-46e5-9ff0-d82d282298b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.150215 May 19 12:08:59.700929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 241/480] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:08:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:00.264739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:00.265388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:00.272492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:00.272964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:00.273729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:00.274157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:00.416843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:00.425860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:00.427488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:00.428684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bd53ea33-8d00-4a55-aeaa-48bba49344cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.168014 May 19 12:09:00.429272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 240/481] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:00.717877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:00.718463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:00.726445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:00.727191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:00.728589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:00.729351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:00.807272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:00.811189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:00.818894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-765caa73-f9cc-4f73-97c3-5582042edd5e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.105036 May 19 12:09:00.819475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 242/482] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:00 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:01.447382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:01.447961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:01.455342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:01.455870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:01.456742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:01.457234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:01.618872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:01.628066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:01.630343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:01.631565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ad37fdbb-2d4b-479b-9271-4f9c97716e5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.188227 May 19 12:09:01.632158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 241/483] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:01.845564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:01.846224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:01.853544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:01.854156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:01.855150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:01.855618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:01.922847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:01.942741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-46b5f292-8ab6-48ca-97de-62718acdad24 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.101591 May 19 12:09:01.942741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 243/484] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:02.650097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:02.651455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:02.658870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:02.659634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:02.660434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:02.660883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:02.809100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:02.818473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:02.819963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:02.821325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-98fb2885-2bb2-4503-8bb6-d22739de0e49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.175274 May 19 12:09:02.821911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 242/485] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:02.952298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:02.953021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:02.959960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:02.960507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:02.961685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:02.962290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:03.031185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:03.039466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-174283fa-65ac-43ab-9655-ef5dfc3748e9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.091066 May 19 12:09:03.040050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 244/486] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:03.839551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:03.840253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:03.847272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:03.847685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:03.848474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:03.849012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:03.989467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:03.998727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:04.000191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:04.001430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-59d8d6e0-ecf7-473a-b791-0d0292e661ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.165806 May 19 12:09:04.001992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 243/487] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:04.061091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:04.061721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:04.068071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:04.068512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:04.069319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:04.069748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:04.132174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:04.141079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-10061d4a-5308-4f9e-9f36-0bd8d5dc0cec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.087326 May 19 12:09:04.141708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 245/488] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:04 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:05.020735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:05.022240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:05.024796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:05.030800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:05.031377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:05.032136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:05.032567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:05.049684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:05.158016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:05.158634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:05.164888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:05.174306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:05.175161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:05.175302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:05.175726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:05.187434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:05.189001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:05.202982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f9117023-5ef6-455c-9510-7ae8d1101784 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.173781 May 19 12:09:05.202982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 244/489] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:05.245134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:05.253974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e030880-3f05-47d9-b3ec-1b609a7de862 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.099441 May 19 12:09:05.260152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 246/490] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:09:06.210289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:06.210918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:06.217688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:06.218143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:06.218898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:06.219314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:06.271346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:06.271925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:06.281731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:06.281731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:06.282627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:06.283208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:06.355562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:06.362760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:06.366705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28cef1e5-cedc-4ddb-8ee0-36a5c08af925 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1222 microversion: 2.1 time: 0.099288 May 19 12:09:06.371586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 247/491] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1222 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:09:06.371760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:06.373039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:06.374205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9eba3279-f46c-49ba-8414-96f1bd68d16b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.167892 May 19 12:09:06.374757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 245/492] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:07.383958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:07.384574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:07.391102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:07.391532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:07.392287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:07.392752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:07.393246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:07.393849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:07.395952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:07.396679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:07.402541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:07.403018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:07.403798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:07.404299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:07.426439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:07.461575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:07.469650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-783d645e-fa88-4587-b02c-7c746a5c62ae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1214 microversion: 2.1 time: 0.089602 May 19 12:09:07.470325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 248/493] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1214 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:07.547950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:07.557772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:07.559323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:07.560550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c63d0a0e-221f-4127-87a8-a3850772ad38 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.171548 May 19 12:09:07.561431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 246/494] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:08.486975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:08.488256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:08.494549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:08.494978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:08.496012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:08.496666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:08.569965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:08.579293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8a910509-c953-496c-adaa-fb961cec6a94 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1288 microversion: 2.1 time: 0.096123 May 19 12:09:08.580104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 249/495] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1288 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:08.594200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:08.594200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:08.602986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:08.604084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:08.605437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:08.606139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:08.981965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:08.991087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:08.992595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:08.993773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1ac1196e-b4df-4091-a325-19769d782581 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.416778 May 19 12:09:08.994349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 247/496] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:09.598840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:09.599417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:09.605601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:09.606043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:09.606802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:09.607226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:09.742609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:09.750234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d73dda4b-29ba-4452-9659-5a1af8864557 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.156640 May 19 12:09:09.750819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 250/497] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:09 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:10.014186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:10.014814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:10.022048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:10.022896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:10.023838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:10.024350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:10.231097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-5cff75fd-3503-45fd-9945-a7d449bfdbcb req-ecd36c3a-46c6-44dd-ab02-b87fe2d21e60 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e2382eba-ee5f-4026-8253-b5835cfb84c4", "tag": "547bc25f-fbc2-430e-b777-1456f40c4c56"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:10.240069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-5cff75fd-3503-45fd-9945-a7d449bfdbcb req-ecd36c3a-46c6-44dd-ab02-b87fe2d21e60 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:10.240857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-5cff75fd-3503-45fd-9945-a7d449bfdbcb req-ecd36c3a-46c6-44dd-ab02-b87fe2d21e60 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:10.254649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:10.260046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-5cff75fd-3503-45fd-9945-a7d449bfdbcb req-ecd36c3a-46c6-44dd-ab02-b87fe2d21e60 service nova] Creating event network-changed:547bc25f-fbc2-430e-b777-1456f40c4c56 for instance e2382eba-ee5f-4026-8253-b5835cfb84c4 on hv-cinder-80403 May 19 12:09:10.269172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:10.271072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:10.272214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6b3c0458-6895-471c-a3f0-a1235ce6a613 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.262531 May 19 12:09:10.272847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 248/498] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:10.274743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-5cff75fd-3503-45fd-9945-a7d449bfdbcb req-ecd36c3a-46c6-44dd-ab02-b87fe2d21e60 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047640 May 19 12:09:10.275343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 251/499] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:09:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:09:10.767893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:10.768541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:10.775304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:10.775753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:10.776516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:10.777002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:10.916142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:10.923945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de6dee6d-e351-435a-ae6c-3593aea40f52 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.160553 May 19 12:09:10.924995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 252/500] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:11.293578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:11.294192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:11.301647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:11.302243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:11.302880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:11.303227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:11.434183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:11.442733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:11.444221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:11.445400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e95b04b-9db1-4b20-8d25-18cdd03bb6d3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.155868 May 19 12:09:11.445988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 249/501] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:11.950676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:11.951513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:11.960418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:11.960913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:11.961666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:11.962090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:12.098300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:12.105936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-171388db-d398-4db3-8342-b877eadcb59c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.162191 May 19 12:09:12.106512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 253/502] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:12.463869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:12.464427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:12.471624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:12.471997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:12.472814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:12.474163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:12.622032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:12.631220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:12.632946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:12.634117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1b6fc94b-1406-47c0-8135-53f424dd999b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.174305 May 19 12:09:12.634690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 250/503] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:13.124513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:13.125214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:13.132666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:13.133146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:13.133900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:13.134328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:13.268818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:13.276885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-620e1f95-62ca-4810-a588-c58f8be21d2f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.156871 May 19 12:09:13.277840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 254/504] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:13 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:13.652586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:13.653217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:13.660193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:13.660672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:13.661463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:13.661889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:13.805534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:13.814271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:13.815777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:13.817004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-03d2fcc5-d774-422d-974f-65f04f7f10f1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.168341 May 19 12:09:13.817559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 251/505] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:14.295248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:14.295825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:14.303170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:14.303636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:14.304442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:14.304940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:14.443796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:14.452090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28884e25-2fcd-4601-a150-3ee26ecc7afa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.161532 May 19 12:09:14.452656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 255/506] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:14.835544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:14.836202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:14.843050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:14.843481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:14.844247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:14.844671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:14.869735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-771cb47d-27b7-48c2-982f-ec1bc169f37b req-5044910a-169b-4877-9555-ad00bfa1cbd7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2382eba-ee5f-4026-8253-b5835cfb84c4", "name": "network-vif-plugged", "status": "completed", "tag": "547bc25f-fbc2-430e-b777-1456f40c4c56"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:14.878056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-771cb47d-27b7-48c2-982f-ec1bc169f37b req-5044910a-169b-4877-9555-ad00bfa1cbd7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:14.878484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-771cb47d-27b7-48c2-982f-ec1bc169f37b req-5044910a-169b-4877-9555-ad00bfa1cbd7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:14.905713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-771cb47d-27b7-48c2-982f-ec1bc169f37b req-5044910a-169b-4877-9555-ad00bfa1cbd7 service nova] Creating event network-vif-plugged:547bc25f-fbc2-430e-b777-1456f40c4c56 for instance e2382eba-ee5f-4026-8253-b5835cfb84c4 on hv-cinder-80403 May 19 12:09:14.911319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-771cb47d-27b7-48c2-982f-ec1bc169f37b req-5044910a-169b-4877-9555-ad00bfa1cbd7 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045106 May 19 12:09:14.912595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 256/507] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:09:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:09:15.008090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:15.017392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:15.019085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:15.020241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-51a403e7-65f5-48dc-a483-63b4702bfce5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.188773 May 19 12:09:15.020848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 252/508] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:15.469989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:15.470595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:15.477288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:15.477731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:15.478499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:15.478971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:15.560050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a01c6463-67e6-43b1-b807-547d04215062"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:15.561275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:15.567462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:15.567901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:15.568707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:15.569328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:15.611259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:15.618760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3e1c507b-8db7-4df2-9052-0b8e49331d3d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.153203 May 19 12:09:15.619502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 257/509] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:16.043113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:16.043991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:16.054973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:16.055647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:16.057037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:16.057740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:16.194731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:16.203315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:16.204821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:16.206019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c4550240-9a29-4fc9-a3f4-8cf646913b8c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.169584 May 19 12:09:16.206580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 258/510] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:09:16.478119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cbaa8c5b-a92a-4c84-b492-11c5dd93cfbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.922757 May 19 12:09:16.478849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 253/511] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:09:15 2022] POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 194 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:09:16.635732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:16.636330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:16.642699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:16.643131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:16.643883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:16.644304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:16.782003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:16.789789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9b6bf65d-db8c-43d2-ad05-4522b72f9bfb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1318 microversion: 2.1 time: 0.158077 May 19 12:09:16.790361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 259/512] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1318 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:16.927582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-585a1338-33e5-4d91-abb3-1c1de0c5a401 req-e9a4e8c9-dd8a-4124-b3a8-c2f3f0c1d767 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "name": "network-vif-plugged", "status": "completed", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:16.936068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-585a1338-33e5-4d91-abb3-1c1de0c5a401 req-e9a4e8c9-dd8a-4124-b3a8-c2f3f0c1d767 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:16.936502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-585a1338-33e5-4d91-abb3-1c1de0c5a401 req-e9a4e8c9-dd8a-4124-b3a8-c2f3f0c1d767 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:16.947511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-585a1338-33e5-4d91-abb3-1c1de0c5a401 req-e9a4e8c9-dd8a-4124-b3a8-c2f3f0c1d767 service nova] Creating event network-vif-plugged:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:09:16.952587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-585a1338-33e5-4d91-abb3-1c1de0c5a401 req-e9a4e8c9-dd8a-4124-b3a8-c2f3f0c1d767 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029547 May 19 12:09:16.953290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 254/513] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:09:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:09:17.224623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:17.225264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:17.231364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.231808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:17.234102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.234102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:17.374189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:17.379559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:17.387730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:17.389775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:17.391181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3c5c2a6f-e8b2-4d76-94b8-4ace2f45377b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.170439 May 19 12:09:17.391798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 260/514] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:17.809843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:17.810652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:17.818311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.818860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:17.819745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.820251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:17.956946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:17.965077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c8f1fa2-ada7-4c53-83d6-e7733dbd5a78 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1508 microversion: 2.1 time: 0.160916 May 19 12:09:17.965665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 255/515] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1508 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:17.982226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "08346d65-e7df-486d-baeb-57ee3caa61df", "device": "/dev/sdb"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:17.983945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:17.990806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.991284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:17.992120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:17.992575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:18.290618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62d88dea-cb81-4ddc-875f-90d32729a2c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "POST /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.312363 May 19 12:09:18.291240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 261/516] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:09:17 2022] POST /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments => generated 194 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:09:18.412499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:18.413127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:18.420256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:18.420689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:18.421465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:18.421934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:18.562127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:18.571512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:18.573110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:18.574415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b48efee-f4f0-4443-b3ff-04b19b91fbfc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.167349 May 19 12:09:18.575081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 256/517] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:19.599532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:19.600230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:19.602201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=56 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:19.608255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:19.608743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:19.609497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:19.609926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:19.749422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:19.758294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:19.759757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:19.760957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b6d90ab-95e5-4d49-b14d-f7f54277ff7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.169275 May 19 12:09:19.761510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 262/518] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:20.779582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:20.780189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:20.789705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:20.790429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:20.791187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:20.791612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:20.937327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:20.946533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:20.948347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:20.949690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e79cdfab-b84f-4357-b1e6-b4b5a4528482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1806 microversion: 2.20 time: 0.174293 May 19 12:09:20.950348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 257/519] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1806 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:21.967559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:21.968143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:21.974884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:21.975323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:21.976067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:21.976486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:22.110298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:22.120629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:22.122888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:22.124499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-53f07eb2-368a-49f9-a472-e8542506edd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1793 microversion: 2.20 time: 0.160806 May 19 12:09:22.125317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 263/520] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:09:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1793 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:09:25.298112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:09:25.299138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:25.305733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:25.306269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:25.307005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:25.307509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:25.330127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'flavor' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:09:25.449577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=4,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:25.475548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3962704f-6203-4510-a289-fba1defa7593 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action" status: 200 len: 32 microversion: 2.1 time: 0.181074 May 19 12:09:25.476208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 258/521] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:09:25 2022] POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action => generated 32 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 19 12:09:25.487076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:25.488102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:25.498430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:25.498985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:25.500140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:25.500812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:25.638050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:25.647767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-08c2a5d5-aae8-4724-add8-1354e2c9b8f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165682 May 19 12:09:25.648564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 264/522] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:26.667021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:26.667663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:26.674037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:26.674487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:26.675237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:26.675657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:26.813305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:26.822050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa0e2991-39c8-469d-8d35-9e461e611347 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160134 May 19 12:09:26.822947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 259/523] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:27.839239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:27.839971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:27.847007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:27.847462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:27.848203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:27.848633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:27.979756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:27.988630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c5ece49f-d7f1-4377-8071-1c4889e40138 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.153232 May 19 12:09:27.989229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 265/524] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:29.007395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:29.007980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:29.015400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:29.015842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:29.016617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:29.017077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:29.152539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:29.161648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e9ce7c5b-6c5d-4234-a7e0-1acd42945758 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.158845 May 19 12:09:29.162374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 260/525] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:30.179073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:30.179811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:30.188235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:30.188969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:30.189547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:30.189971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:30.338960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:30.351715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e09ad583-223f-42aa-9f1d-b00e192f71ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.175620 May 19 12:09:30.353851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 266/526] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:31.370768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:31.371520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:31.378750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:31.379189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:31.379939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:31.380575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:31.526401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:31.534686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-73c48914-d372-4edc-87a2-1dddc8f81554 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168278 May 19 12:09:31.535259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 261/527] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:32.551653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:32.552259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:32.559199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:32.559642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:32.560410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:32.560869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:32.689735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:32.698146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b0da5cdc-6529-4d3d-9a3b-b35eb4458569 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.150364 May 19 12:09:32.698731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 267/528] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:33.718944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:33.719653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:33.728545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:33.729141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:33.730115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:33.730649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:33.868623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:33.878365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36a14223-33cb-41d6-8d97-9137409afd5e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165276 May 19 12:09:33.878935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 262/529] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:34.894964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:34.895567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:34.902470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:34.902920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:34.903675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:34.904101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:35.039975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:35.048177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-74c4280b-0b8d-40d8-89aa-3656b8f482f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.157194 May 19 12:09:35.048761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 268/530] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:36.066044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:36.066730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:36.069360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=38 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:36.075265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:36.076333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:36.077106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:36.077541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:36.145600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:36.146188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:36.152940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:36.153486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:36.154304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:36.154736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:36.220186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:36.228578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-30fe0264-ab54-41b1-9737-5f2a4e44352b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166520 May 19 12:09:36.229179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 263/531] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:36.279909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'services' on Instance uuid 4518b411-8fac-48f2-8826-7b91651826b1 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:09:36.397028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de832b78-26a8-4132-85c3-dcbee008afe9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments/6f43fdb8-cc39-4d84-bce4-ec8a02bfa85b" status: 202 len: 0 microversion: 2.1 time: 0.255232 May 19 12:09:36.397699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 269/532] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:09:36 2022] DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments/6f43fdb8-cc39-4d84-bce4-ec8a02bfa85b => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:09:37.246369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:37.247115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:37.266537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:37.267045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:37.268913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:37.269486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:37.407048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:37.421457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86408c1f-e46b-4dab-8be4-ee89122e6158 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.177940 May 19 12:09:37.422754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 264/533] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:38.442268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:38.442949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:38.450118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:38.450555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:38.451305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:38.451725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:38.590065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:38.598220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-71fba69a-e3a1-4fab-b8fd-11e0e0118956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161234 May 19 12:09:38.598772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 270/534] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:39.615456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:39.616180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:39.630860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:39.631375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:39.632211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:39.632754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:39.762904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:39.770594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fd6c8c04-c6a1-4533-b6ec-5f7900be8af8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159114 May 19 12:09:39.771164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 265/535] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:40.793073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:40.793910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:40.801748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:40.802198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:40.802963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:40.803392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:40.937184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:40.945760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5c747d95-731c-4849-9624-b55cd42ef169 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160308 May 19 12:09:40.946300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 271/536] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:40 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:41.964687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:41.965245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:41.972792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:41.973233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:41.973989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:41.974416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:42.115733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:42.124406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-33bd5b28-d17c-4ed5-9fab-fed34ddf44aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164279 May 19 12:09:42.125030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 266/537] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:43.146219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:43.147220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:43.163274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:43.164017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:43.165330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:43.166042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:43.309461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:43.321196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-26bd798b-1554-44f4-a562-8dd947a8c62a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.181468 May 19 12:09:43.321992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 272/538] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:44.340822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:44.341519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:44.348905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:44.349346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:44.350097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:44.350521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:44.492172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:44.500773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e8f358e4-05a2-493b-b910-9a99388c7e16 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164711 May 19 12:09:44.501596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 267/539] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:45.519239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:45.519840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:45.526745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:45.527182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:45.527900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:45.528352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:45.679272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:45.689758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6fee4345-b1b9-4f01-850c-a91eaca2df54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.174581 May 19 12:09:45.690334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 273/540] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:46.710946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:46.711880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:46.721192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:46.721634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:46.722380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:46.722801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:46.873351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:46.882413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aed362d5-9dfc-427a-83d5-568121bc90c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.178020 May 19 12:09:46.883033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 268/541] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:47.903576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:47.904513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:47.912182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:47.912620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:47.913392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:47.913812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:47.916461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:48.067280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:48.079734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-79b02115-34ae-4d46-ba4c-09d9786828f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.182470 May 19 12:09:48.080484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 274/542] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:49.099064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:49.099893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:49.110015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:49.110465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:49.111223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:49.111695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:49.246641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:49.255092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-62b735bd-bb55-49ca-b707-9fa9e18de617 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161776 May 19 12:09:49.255722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 269/543] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:50.274478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:50.275097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:50.277043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=26 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:50.282897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:50.283329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:50.284080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:50.284500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:50.434516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:50.442749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0dd4b0ea-3213-4d0b-b60a-f850813e5c6c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172235 May 19 12:09:50.443317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 275/544] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:51.465225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:51.465820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:51.472789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:51.473233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:51.473981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:51.474402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:51.615701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:51.628205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5184b592-29c0-4dbd-84be-fc63f9949efb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166985 May 19 12:09:51.628803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 270/545] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:52.649703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:52.650745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:52.660430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:52.660909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:52.661669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:52.662097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:52.804282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:52.815293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c13bba9e-70fa-4f9a-9817-97f9a8cdbdf2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171952 May 19 12:09:52.816156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 276/546] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:53.842199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:53.843151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:53.851128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:53.851563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:53.852308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:53.852761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:53.994333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:54.003379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45b44889-2bde-4448-a1bc-bb87945a431c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167685 May 19 12:09:54.003955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 271/547] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:55.030451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:55.031558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:55.043978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:55.044718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:55.045992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:55.046832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:55.190554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:55.203045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-858b6a9e-41cb-4aa3-91d9-ad90ddd89e4c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.178809 May 19 12:09:55.203867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 277/548] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:56.221384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:56.221995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:56.229005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:56.229440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:56.230194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:56.230613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:56.233169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=81,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:56.363763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:56.372259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c6575d6-8c64-40be-b829-f76a2dde9885 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154890 May 19 12:09:56.372911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 272/549] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:57.390181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:57.390776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:57.397867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:57.398316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:57.399066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:57.399495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:57.538526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:57.547262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0a185a65-1518-4522-93d7-a42570d85489 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160940 May 19 12:09:57.547852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 278/550] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:58.564956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:58.565570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:58.572492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:58.572964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:58.573719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:58.574148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:58.721270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:58.729395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b9de6227-e5b1-4fa2-9a7d-64e5115a3b4e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168571 May 19 12:09:58.730182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 273/551] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:09:59.747577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:09:59.748183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:09:59.755850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:59.756753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:59.757772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:09:59.758204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:09:59.898422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:09:59.906871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0aa72294-04e1-4e8b-9f64-2cfeaeea015c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163347 May 19 12:09:59.907422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 279/552] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:09:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:00.933330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:00.934330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:00.945471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:00.946179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:00.947463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:00.948171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:01.087490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:01.095678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e9a17cc-6bbc-430a-96df-ae3efdbff80c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168837 May 19 12:10:01.096252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 274/553] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:02.117286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:02.118172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:02.128525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:02.129001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:02.129778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:02.130201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:02.276329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:02.294010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ec7ebd73-9adc-4ac4-82fe-44aae4cb0062 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.183305 May 19 12:10:02.294583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 280/554] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:03.311425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:03.312069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:03.319184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.319659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.320419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.320897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.459708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:03.467719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6525f472-54ea-4121-9b91-61093bc0fa59 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160243 May 19 12:10:03.468296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 275/555] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:03.649153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:03.652451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:03.661514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.664787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.665530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.665958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.695574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-438defb9-0cdd-4f49-8f47-7aada55826a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments/657bcb07-4e79-4f43-b97d-9210968ea599" status: 200 len: 194 microversion: 2.20 time: 0.053555 May 19 12:10:03.696157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 281/556] 10.222.0.35 () {60 vars in 1453 bytes} [Thu May 19 12:10:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments/657bcb07-4e79-4f43-b97d-9210968ea599 => generated 194 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:03.765386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:03.765850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:03.772243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.772675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.773479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.773875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.870124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lazy-loading 'services' on Instance uuid b642ea31-4427-475b-8374-591f4dcbadc8 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:10:03.964412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65ad320f-b168-401c-a23d-17fc8e6b9912 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments/657bcb07-4e79-4f43-b97d-9210968ea599" status: 202 len: 0 microversion: 2.20 time: 0.202918 May 19 12:10:03.965060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 276/557] 10.222.0.35 () {60 vars in 1456 bytes} [Thu May 19 12:10:03 2022] DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments/657bcb07-4e79-4f43-b97d-9210968ea599 => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 19 12:10:03.975204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:03.976544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:03.983977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.984565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:03.985606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:03.986299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:04.019879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5eae4286-e1b4-41d5-aca9-576b22124193 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049918 May 19 12:10:04.020449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 282/558] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 12:10:04.487223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:04.488346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:04.497662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:04.498241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:04.499129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:04.499564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:04.640253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:04.648690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e893e367-1bf2-4cfb-833c-c9483038a763 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167232 May 19 12:10:04.649298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 277/559] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:05.037220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:05.038137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:05.044989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:05.046152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:05.047006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:05.047516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:05.081638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-59bbce6a-a298-4b3e-8871-e0319d0a6bad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046500 May 19 12:10:05.082228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 283/560] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:05.665680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:05.666287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:05.673492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:05.673932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:05.674735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:05.675158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:05.815542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:05.824348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d749b962-b0b9-4fb7-9ce1-f71df2a6a662 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162598 May 19 12:10:05.824993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 278/561] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:06.097734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:06.098678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:06.106354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:06.106834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:06.107693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:06.108145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:06.139884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9560de01-a410-4d2d-8395-27bb95df51cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048088 May 19 12:10:06.140477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 284/562] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:06.843133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:06.843840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:06.846648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=32 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:06.853172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:06.853687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:06.854575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:06.855075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:06.987858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:06.996414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b3e438f0-bfee-4069-9f8a-54407015dd47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.157820 May 19 12:10:06.997028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 279/563] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:07.156097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:07.156993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:07.166935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:07.167544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:07.168705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:07.169172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:07.200125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-134efccf-d41f-41ed-97b2-92179df25cf8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050422 May 19 12:10:07.200686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 285/564] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:08.015693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:08.016487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:08.023750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:08.024192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:08.025443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:08.026136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:08.187247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:08.196821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-083880f8-a0fe-44d5-b646-e276f88dc5e1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.186477 May 19 12:10:08.197453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 280/565] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:08.211937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:08.212922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:08.220103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:08.220917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:08.221668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:08.222088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:08.254783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6496f322-4763-4322-85c5-b4e05a5c7a30 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046920 May 19 12:10:08.255380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 286/566] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:09.218781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:09.219758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:09.231250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:09.232045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:09.233401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:09.234120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:09.266321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:09.267259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:09.274420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:09.274854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:09.275618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:09.276042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:09.321421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-45df89a6-4eb4-4f48-aba5-98c62ad4be7c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050702 May 19 12:10:09.326147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 287/567] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:09.379785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:09.387761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bd0ed049-59b0-4af7-90eb-ce54d3b1258e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.175612 May 19 12:10:09.388326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 281/568] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:10.334937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:10.336001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:10.344563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:10.345213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:10.346108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:10.346612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:10.377223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2c99c87a-0962-483e-a2c6-c06f3d457c55 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047086 May 19 12:10:10.377979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 288/569] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:10.404964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:10.405637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:10.412311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:10.412782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:10.413529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:10.413936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:10.545933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:10.554109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e6f5369-e0f3-42bc-8990-fed978e18b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.153090 May 19 12:10:10.554659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 282/570] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:11.394562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:11.395362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:11.402438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:11.402885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:11.403637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:11.404091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:11.437657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d6f70e92-692b-41ff-bd5f-16ece9bed031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050088 May 19 12:10:11.438239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 289/571] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:11.574178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:11.575077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:11.584606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:11.585075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:11.585834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:11.586265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:11.729622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:11.741610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee603e4a-b27e-431b-b5ec-2d25083fe03a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173383 May 19 12:10:11.742384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 283/572] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:12.450193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:12.451099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:12.457846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:12.458311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:12.459047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:12.459466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:12.483876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:12.490936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fdde0a8d-ed2a-4b5b-ac06-63f7cbc55191 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044869 May 19 12:10:12.491501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 290/573] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:12.764389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:12.765331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:12.776488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:12.777290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:12.778596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:12.779301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:12.919741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:12.928369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-33c24846-6267-4f0d-9f61-bbc473c9039c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170817 May 19 12:10:12.928980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 284/574] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:13.502874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:13.503804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:13.510544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:13.510990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:13.511745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:13.512171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:13.542501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7208b8cc-633e-4b21-85f1-9c2d14f0e52e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043961 May 19 12:10:13.543072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 291/575] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:13.950025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:13.950637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:13.957740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:13.958183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:13.958939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:13.959364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:14.090009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:14.098602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ef10ec5-cd1b-4f4a-bba3-e41551d06391 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.152462 May 19 12:10:14.099183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 285/576] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:14.555119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:14.556074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:14.563021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:14.563501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:14.564258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:14.564688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:14.593972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7b52fa51-8e6b-41c5-a6e4-68e4df1a8c5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042973 May 19 12:10:14.594528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 292/577] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:15.118149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:15.118721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:15.126465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:15.126899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:15.127673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:15.128125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:15.266080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:15.274688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cb7c24d6-19be-4b24-8ac0-c30dd195e13d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161920 May 19 12:10:15.275251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 286/578] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:15.282487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:10:15.282642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:10:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:10:15.606434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:15.607342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:15.614496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:15.614934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:15.615674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:15.616101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:15.646891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-162f0deb-7159-4edf-9c00-25261ac0b580 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044554 May 19 12:10:15.647468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 293/579] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:16.296369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:16.297273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:16.308189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:16.308979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:16.310207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:16.310874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:16.467113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:16.475803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e0ddb0d-2ce2-4d04-aac0-c326847481a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.185956 May 19 12:10:16.476374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 287/580] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:16.663212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:16.664731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:16.675686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:16.676128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:16.676916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:16.677343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:16.707178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3aac7b0e-df1f-403a-a54d-a337d1c463a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050817 May 19 12:10:16.707740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 294/581] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:17.494726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:17.495481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:17.504381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.504990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.505924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.506613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.510044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:17.662989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:17.674910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9dec2945-d391-4de3-a2bd-374c1249446c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.184932 May 19 12:10:17.675668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 288/582] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:17.719140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:17.720058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:17.727151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.727595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.728352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.728826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.758619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-73cd8371-9663-4d3d-adcf-efaba85bc110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043476 May 19 12:10:17.759158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 295/583] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:17.807182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:17.807704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:17.814186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.814629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.815412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.815881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.838578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lazy-loading 'flavor' on Instance uuid 08f420db-5800-43e3-9990-bd7aa43a9d3e {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:10:17.923317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-054918bc-3abf-460d-bb5d-d3127efbeaf5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "POST /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e/action" status: 202 len: 0 microversion: 2.1 time: 0.120600 May 19 12:10:17.923883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 289/584] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:10:17 2022] POST /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e/action => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:10:17.934776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:17.935728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:17.944306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.944779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:17.945522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:17.945935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:18.076365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:18.085548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1527e913-f5dc-44c1-afd2-15553acee195 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.156703 May 19 12:10:18.086117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 296/585] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:18.696387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:18.697269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:18.705309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:18.705853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:18.706773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:18.707309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:18.770592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:18.771473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:18.777768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:18.778210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:18.778967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:18.779394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:18.809259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d624011b-bd2e-48d4-be47-a2282c164832 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042675 May 19 12:10:18.809869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 297/586] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:18.849371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:18.857539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8f610ce7-5205-42f0-9a98-6ed6e762ae0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167321 May 19 12:10:18.858242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 290/587] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:19.105240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:19.105919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:19.113802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.114242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:19.114998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.115423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:19.260219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:19.271607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ec5d2f37-97af-47d7-8f0d-7fcb98b28c1f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170498 May 19 12:10:19.272460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 298/588] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:19.825537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:19.827137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:19.835752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.836187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:19.836974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.837398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:19.869100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a831c819-667c-4be3-85e6-21c5c11613f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050499 May 19 12:10:19.871942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 291/589] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:19.876432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:19.877110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:19.883785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.884224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:19.885010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:19.885429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:20.025053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:20.036595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1b95c136-4ab2-4325-85e3-2d51becc7708 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164156 May 19 12:10:20.037197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 299/590] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:20.296553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:20.297688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:20.307839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:20.308566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:20.309847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:20.310586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:20.627004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:20.638134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-480cb410-ce19-472b-97b2-79537bd78ae7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.347699 May 19 12:10:20.638907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 292/591] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:20.882628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:20.883733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:20.887219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=50 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:20.895720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:20.896263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:20.897293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:20.897893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:20.931773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-66206fc9-ad51-47de-97ba-7ce2dfae546f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054147 May 19 12:10:20.932367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 300/592] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:21.057808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:21.058845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:21.068306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.068783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.069537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.069963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.220184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:21.228587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a070fc41-8194-4082-baf8-7314ddf3ffdf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.177370 May 19 12:10:21.229157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 293/593] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:21.658797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:21.659553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:21.666267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.666718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.667472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.667897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.802049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:21.809690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b468c242-f328-4760-8271-0cb59a250b82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.155705 May 19 12:10:21.810263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 301/594] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:21.944752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:21.945706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:21.953073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.953606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.954501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:21.955041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:21.991289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0d1feca-d2a1-47e2-b7d7-a5bfefe64565 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050558 May 19 12:10:21.991954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 294/595] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:22.245873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:22.246498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:22.253772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:22.254262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:22.255012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:22.255463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:22.454416 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:22.465824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-db5fb2bd-2a22-4203-904c-b0365b3a0d99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.223576 May 19 12:10:22.466570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 302/596] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:22.830782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:22.831771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:22.838922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:22.839363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:22.840125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:22.840551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:22.975835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:22.985911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-68f291e8-bd0d-4d94-80af-524548aed117 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161851 May 19 12:10:22.986458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 295/597] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:23.004524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:23.005496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:23.011676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:23.012112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:23.012908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:23.013330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:23.043677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2eb3e711-1db1-4a8b-bb13-8d716e9d52e7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043210 May 19 12:10:23.044252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 303/598] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 12:10:23.487148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:23.487739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:23.495109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:23.495550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:23.496285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:23.496725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:23.638614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:23.647156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28dcb90a-3ff7-47d2-92a4-92775ee2dc03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164664 May 19 12:10:23.647736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 296/599] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:24.011470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:24.012500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:24.023485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.024245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.025544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.026257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.055972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:24.057110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:24.064024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.064505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.065438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.065848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.097027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36b6df72-2461-499b-9d5f-624d9e50fc3b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044925 May 19 12:10:24.097622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 297/600] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:24.171695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:24.179850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6f42bd83-64e3-4826-b500-ae41f8258335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.175011 May 19 12:10:24.180440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 304/601] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:24.664935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:24.665538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:24.672051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.672496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.673287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:24.673717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:24.808357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:24.816675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2bd842f4-a95a-4018-a43e-25bbe27cbec6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155770 May 19 12:10:24.817227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 298/602] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:25.109191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:25.110127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:25.117150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.117597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.118353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.118781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.150180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-938c0968-ae44-4f49-837f-7b8aab81a649 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045030 May 19 12:10:25.150757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 305/603] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:25.200803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:25.200803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:25.204196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.204658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.205418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.205985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.349974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:25.358391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-adb456ef-1399-4754-99be-6481a3d22382 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164917 May 19 12:10:25.358959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 299/604] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:25.838096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:25.838899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:25.847364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.847947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.848979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:25.849537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:25.989943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:26.003522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d97d4812-308b-4efb-abd6-ea0e9a3d38d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171752 May 19 12:10:26.004444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 306/605] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:26.166440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:26.167783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:26.178037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:26.178776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:26.180015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:26.180727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:26.215178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4b207f3f-f5ac-4aec-b1f4-9a775a98701b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055511 May 19 12:10:26.215845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 300/606] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:26.379737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:26.380725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:26.391795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:26.392531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:26.393853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:26.394570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:26.409743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:26.527745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:26.535722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e27b3a60-b36b-415b-ab9d-604b78af5801 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162900 May 19 12:10:26.536294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 307/607] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:27.025596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:27.026451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:27.036689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.037443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.038681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.039392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.177773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:27.187120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c41c5597-6690-49ab-a999-498ac7ce2fbd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168163 May 19 12:10:27.187716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 301/608] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:27.227661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:27.228602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:27.234784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.235221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.235959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.236379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.266515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f6ac7833-5587-408e-86c2-387bb6ddd9e6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042896 May 19 12:10:27.267071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 308/609] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:27.555548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:27.556482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:27.569242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.569713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.570496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:27.570934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:27.714144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:27.716539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:27.725780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bf8c15b9-9636-4f46-8f29-3899d1f64eef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.176177 May 19 12:10:27.726373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 302/610] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:28.207634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:28.208171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.218040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.218640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.219666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.220238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.279246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:28.280182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.287830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.288271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.289079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.289501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.320892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-948d0bfe-cae9-471c-aae7-aedc8addb88c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046371 May 19 12:10:28.321778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 303/611] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:28.355169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:28.363586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-93cdb532-90eb-4f49-9a92-c6789c7ca98f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1717 microversion: 2.1 time: 0.161771 May 19 12:10:28.364371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 309/612] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1717 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:28.522433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:28.522921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.529109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.529542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.530285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.530706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.619070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Cannot 'detach_volume' instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 while it is in vm_state rescued May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:10:28.620735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:10:28.622305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Cannot 'detach_volume' instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 while it is in vm_state rescued May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 in vm_state rescued. Cannot detach_volume while the instance is in this state. May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:10:28.622976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 while it is in vm_state rescued May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 while it is in vm_state rescued May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:10:28.624873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 409 to user: Cannot 'detach_volume' instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 while it is in vm_state rescued {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:10:28.626077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6560af4d-e400-42cc-9509-8aaf4940ee02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 409 len: 152 microversion: 2.1 time: 0.158635 May 19 12:10:28.626077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 304/613] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:10:28 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 152 bytes in 160 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 19 12:10:28.632127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:28.632695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.643785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.644206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.645296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.645712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.718723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3ea7feb0-9ed9-477b-bb1f-3ad02783553f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action" status: 202 len: 0 microversion: 2.1 time: 0.090408 May 19 12:10:28.719597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 310/614] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:10:28 2022] POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:10:28.727778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:28.728389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.734795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.735239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.736003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.736428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.744974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:28.746046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:28.753697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.754280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.755046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:28.755512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:28.878561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:28.886743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b414fbae-a145-4bb0-bd37-b86471c15691 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1725 microversion: 2.1 time: 0.162847 May 19 12:10:28.887318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 305/615] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1725 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:28.961681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:28.971190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f5cec965-84bb-4333-8ab0-ef30d7fcdf95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.230026 May 19 12:10:28.971771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 311/616] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:29.333309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:29.334346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:29.340810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.341245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:29.341989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.342415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:29.373067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-304b98b3-1b65-4e77-9bd6-96807350b228 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043989 May 19 12:10:29.373614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 306/617] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:29.907189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:29.908239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:29.916738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.916985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:29.917775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.918195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:29.988786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:29.989384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:29.996443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.996916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:29.997665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:29.998089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:30.070121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:30.078712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba00966e-b206-4920-92e8-7445749d5ce3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1725 microversion: 2.1 time: 0.177598 May 19 12:10:30.084731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 312/618] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1725 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:30.155285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:30.166712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-50babd93-586b-46e4-832d-556243f92f7d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.181797 May 19 12:10:30.167282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 307/619] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:30.385305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:30.386119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:30.392964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:30.393498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:30.394573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:30.395407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:30.427241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a15ad86-45f4-4e48-b606-c23edcdb4bae tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046071 May 19 12:10:30.427802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 313/620] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:31.099643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:31.100495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:31.111584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.112313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.113593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.114308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.189425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:31.190363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:31.197855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.198301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.199067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.199487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.253308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:31.262077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f79c9f9-11c2-4f79-aee0-741464faabc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1725 microversion: 2.1 time: 0.169120 May 19 12:10:31.262660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 308/621] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1725 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:31.335288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:31.342966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eff2182f-1bff-4d12-abdb-14e0449754e0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.159782 May 19 12:10:31.343542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 314/622] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:31.439381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:31.440346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:31.446930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.447378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.448156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:31.448765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:31.482484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cb2b5858-f48f-4f71-ae50-308b69ff5d0a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047205 May 19 12:10:31.483061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 309/623] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:32.280452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:32.281279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:32.288971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.289407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.290154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.290578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.359674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:32.360285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:32.367385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.367828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.368589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.369051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.422439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:32.431043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0c209f08-3db2-4242-9ce5-5879ddb6fbe4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1725 microversion: 2.1 time: 0.155181 May 19 12:10:32.431705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 315/624] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:32.495063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:32.495966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:32.502930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:32.503124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.503919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.504994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:32.505697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:32.512665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5485db10-5132-42b2-a783-789499275a22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.156914 May 19 12:10:32.513261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 310/625] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:32.538424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4a5fbb4a-240f-43c2-9789-0d3b8edb0765 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047316 May 19 12:10:32.538902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 316/626] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 12:10:33.452141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:33.453116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:33.464318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.465107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.466360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.467064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.531130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:33.531730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:33.538244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.538829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.539598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.540018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.604268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:33.613884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b6a75aa3-9160-4863-bd3f-870d8e87e1b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1731 microversion: 2.1 time: 0.168248 May 19 12:10:33.614691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 311/627] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1731 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:33.619698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:33.620429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:33.626881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.627475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.628339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.628685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.661370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ed51ebfb-d8bb-4520-a9c4-eae998e42298 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045786 May 19 12:10:33.661937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 312/628] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:33.666475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:33.666948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:33.672584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.673050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.673790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:33.674210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:33.693562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:33.701203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-613bb103-ff66-4a32-b00f-067376e3bcca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174760 May 19 12:10:33.701868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 317/629] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:33.749188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3977f613-7c39-4782-bdd2-e09b2fbd4b07 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.085737 May 19 12:10:33.749800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 313/630] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:10:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-interface => generated 284 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:10:34.623736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-cc4caffa-b04a-46cf-a394-e0123d758d4a req-dda135e9-64c3-4e17-acfb-e7f130a31db9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:34.632610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-cc4caffa-b04a-46cf-a394-e0123d758d4a req-dda135e9-64c3-4e17-acfb-e7f130a31db9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:34.632950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-cc4caffa-b04a-46cf-a394-e0123d758d4a req-dda135e9-64c3-4e17-acfb-e7f130a31db9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:34.644277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-cc4caffa-b04a-46cf-a394-e0123d758d4a req-dda135e9-64c3-4e17-acfb-e7f130a31db9 service nova] Creating event network-changed:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:10:34.649924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-cc4caffa-b04a-46cf-a394-e0123d758d4a req-dda135e9-64c3-4e17-acfb-e7f130a31db9 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029894 May 19 12:10:34.650692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 318/631] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:10:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:10:34.673870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:34.674772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:34.681086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:34.681523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:34.682272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:34.682700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:34.714522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-32131580-eb42-427e-97fb-ef803f839b5a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045234 May 19 12:10:34.718580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 314/632] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:34.723717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:34.724412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:34.731398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:34.732105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:34.732892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:34.733325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:34.932277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:34.941052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa3292c6-96b6-4817-9236-25d08e6bb0c5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.226151 May 19 12:10:34.943149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 319/633] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:35.731779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:35.733132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:35.744371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:35.745151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:35.746380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:35.747105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:35.779927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ab3d5f0a-8fb1-4e03-9581-bbd962233848 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055398 May 19 12:10:35.780490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 315/634] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:35.958217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:35.958748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:35.966083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:35.966519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:35.967261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:35.967680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:36.108801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:36.117144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42ed56f3-b562-4761-b7e9-a8a9e072e848 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162960 May 19 12:10:36.117715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 320/635] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:36.667846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-b5f199b3-b5a6-4c9a-888d-4fdcea29e946 req-6a3f4d13-5db9-47b2-9094-dd214cab9d98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:36.676498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b5f199b3-b5a6-4c9a-888d-4fdcea29e946 req-6a3f4d13-5db9-47b2-9094-dd214cab9d98 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:36.676926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b5f199b3-b5a6-4c9a-888d-4fdcea29e946 req-6a3f4d13-5db9-47b2-9094-dd214cab9d98 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:36.688609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-b5f199b3-b5a6-4c9a-888d-4fdcea29e946 req-6a3f4d13-5db9-47b2-9094-dd214cab9d98 service nova] Creating event network-changed:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:10:36.693881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-b5f199b3-b5a6-4c9a-888d-4fdcea29e946 req-6a3f4d13-5db9-47b2-9094-dd214cab9d98 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029675 May 19 12:10:36.694457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 316/636] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:10:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:10:36.796013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:36.797415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:36.805304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:36.805751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:36.806500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:36.806925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:36.837199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bb9bbd99-8999-41a9-ad22-d08ac3c7307f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047871 May 19 12:10:36.837764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 321/637] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:37.135925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:37.136672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:37.138645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=76 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:37.146243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:37.146684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:37.147442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:37.147873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:37.345703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:37.355895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d51a327a-6422-46cb-bda5-6d407759b029 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.224327 May 19 12:10:37.357242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 317/638] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:37.849332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:37.850396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:37.857317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:37.857766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:37.858521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:37.858951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:37.889090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-65022868-0f82-4e82-882f-19cac2ed9d53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043943 May 19 12:10:37.889666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 322/639] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:38.373256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:38.374196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:38.381302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:38.381749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:38.382513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:38.382965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:38.514412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:38.523305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cab58107-80bd-460c-8cda-0c2d678f4337 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.153839 May 19 12:10:38.523832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 318/640] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:38.901916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:38.902905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:38.910811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:38.911247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:38.911992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:38.912417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:38.937175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:38.943879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-218d3d81-7297-4c13-9347-466698d9fad8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046495 May 19 12:10:38.944431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 323/641] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:39.540982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:39.541542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:39.548589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:39.549056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:39.549794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:39.550218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:39.695853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:39.706938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86175fbb-6e2b-4e45-a58f-dd9540752670 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169794 May 19 12:10:39.707715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 319/642] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:39.957246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:39.958157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:39.965752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:39.966190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:39.966942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:39.967370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:39.997593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fd2439eb-4939-4197-82d4-ff552a5fae68 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045644 May 19 12:10:39.998339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 324/643] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:40.731613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:40.732524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:40.741686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:40.742125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:40.742872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:40.743295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:40.758428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:40.879303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:40.887387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-09667754-0486-4bbc-8738-59e97f0f5f57 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162343 May 19 12:10:40.887961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 320/644] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:41.010331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:41.011202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:41.017890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:41.018321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:41.019113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:41.019381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-18f8304d-8ac7-4f8b-b626-56f1109df39e req-e256d8e0-ceb5-40f4-88f4-7e9a1dafeabf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "name": "network-vif-unplugged", "status": "completed", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:41.019522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:41.028056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-18f8304d-8ac7-4f8b-b626-56f1109df39e req-e256d8e0-ceb5-40f4-88f4-7e9a1dafeabf service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:41.028503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-18f8304d-8ac7-4f8b-b626-56f1109df39e req-e256d8e0-ceb5-40f4-88f4-7e9a1dafeabf service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:41.040400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-18f8304d-8ac7-4f8b-b626-56f1109df39e req-e256d8e0-ceb5-40f4-88f4-7e9a1dafeabf service nova] Creating event network-vif-unplugged:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:10:41.046580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-18f8304d-8ac7-4f8b-b626-56f1109df39e req-e256d8e0-ceb5-40f4-88f4-7e9a1dafeabf service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031177 May 19 12:10:41.047455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 321/645] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:10:41 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:10:41.053832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1b2b7156-415b-42cd-951c-178a5b651751 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047681 May 19 12:10:41.054415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 325/646] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:41.910411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:41.911102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:41.917799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:41.918239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:41.918986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:41.919407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:42.064631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:42.066516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:42.067424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:42.072824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0dcc923c-04cc-44cc-b899-c84fdd71204e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170302 May 19 12:10:42.073420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 322/647] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:42.073685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:42.073980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:42.074732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:42.075155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:42.103925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-585f7c74-4efe-4403-bf4b-3218c0d8ecc2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042072 May 19 12:10:42.104464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 326/648] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:43.090910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:43.091532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:43.099251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:43.099748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:43.100587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:43.101098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:43.227844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:43.229011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:43.233217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:43.235059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:43.235498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:43.236255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:43.236699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:43.240939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-128ad8f6-2750-4005-a85b-2b135c6ba32e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.154449 May 19 12:10:43.241772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 323/649] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:43.269022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1f80a06d-26ae-4c1c-955b-0c30c40f63a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.157260 May 19 12:10:43.269618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 327/650] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:44.263398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:44.264311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:44.274566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.275285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.276573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.277329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.283960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:44.285364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:44.292100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.292541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.293329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.293754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.323591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0c4a161e-05ff-4735-acc4-18ff73bf4a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046171 May 19 12:10:44.324387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 328/651] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:44.411974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:44.419977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c100350e-0811-4dd2-bac2-603dab68f23c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.163842 May 19 12:10:44.420544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 324/652] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:44.747072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:44.747933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:44.756836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.757300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.758041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.758462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.782669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:10:44.863994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f2356ea1-c4c2-4d25-aa23-6aca84279615 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 204 len: 0 microversion: 2.1 time: 0.122959 May 19 12:10:44.864888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 329/653] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:10:44 2022] DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:10:44.873767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:44.874413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:44.880725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.881162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:44.881901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:44.882323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:45.034846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:45.042653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-50261941-c4d8-42bb-b950-8bb4597cc19d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1579 microversion: 2.1 time: 0.173352 May 19 12:10:45.043253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 325/654] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:44 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1579 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:10:45.337000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:45.338017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:45.344899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:45.345458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:45.346218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:45.346648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:45.376456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d27bd4ba-4db7-4c31-89f7-0dc68be5a876 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043791 May 19 12:10:45.377051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 330/655] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:45.440912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:45.441821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:45.449491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:45.449981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:45.450815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:45.451290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:45.588170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:45.599011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f52d5195-d1b0-4290-bc8a-8aaeca0656af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164632 May 19 12:10:45.599582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 326/656] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:46.060327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:46.060933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:46.067656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.068095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.068871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.069295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.218253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:46.226511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-13ed73ae-a98c-4d64-a6c5-c23ceca0e30a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1579 microversion: 2.1 time: 0.170103 May 19 12:10:46.227073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 331/657] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:46 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1579 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:46.391931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:46.393348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:46.403178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.403815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.405102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.405829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.437083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36a16944-86ab-4233-af81-6ef361cad476 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052004 May 19 12:10:46.437640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 327/658] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:46.619845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:46.620869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:46.632030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.632807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.634018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:46.634751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:46.779270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:46.787308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bad29f6b-2269-462a-8811-ba5d2d150781 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174008 May 19 12:10:46.787882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 332/659] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:46 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:47.243431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:47.244051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:47.251470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.251933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.252690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.253136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.449063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:47.450025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:47.455277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:47.456353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.457009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.457778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.458203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.464368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b00ac7fe-ff76-4be8-9df1-4c258ab2c01a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1579 microversion: 2.1 time: 0.224738 May 19 12:10:47.465397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 328/660] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:47 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1579 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:47.488953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-61890aba-ae49-4600-957b-907d5813db5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043782 May 19 12:10:47.489760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 333/661] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:47.730278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-495faa62-d74a-4615-a128-7389c950b200 req-47d1d31e-ea17-4c4e-a1ca-19c865e0cdc6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7b377f1-b40c-4146-8569-44510a0c33ab", "name": "network-vif-deleted", "tag": "1bd50345-4513-45c1-ae8b-577dff600ff2"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:10:47.738375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-495faa62-d74a-4615-a128-7389c950b200 req-47d1d31e-ea17-4c4e-a1ca-19c865e0cdc6 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.738929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-495faa62-d74a-4615-a128-7389c950b200 req-47d1d31e-ea17-4c4e-a1ca-19c865e0cdc6 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.751408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-495faa62-d74a-4615-a128-7389c950b200 req-47d1d31e-ea17-4c4e-a1ca-19c865e0cdc6 service nova] Creating event network-vif-deleted:1bd50345-4513-45c1-ae8b-577dff600ff2 for instance a7b377f1-b40c-4146-8569-44510a0c33ab on hv-cinder-80403 May 19 12:10:47.756942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-495faa62-d74a-4615-a128-7389c950b200 req-47d1d31e-ea17-4c4e-a1ca-19c865e0cdc6 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031117 May 19 12:10:47.758193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 329/662] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:10:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:10:47.805079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:47.805689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:47.812909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.813351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.814119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:47.814541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:47.951286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:47.959229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-22ee4219-cdb4-4ae8-9b96-39aaf49fa9de tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.158665 May 19 12:10:47.959880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 334/663] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:48.483738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:48.484467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:48.492331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.492817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:48.493845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.494275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:48.501128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:48.502385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:48.512396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.513084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:48.514106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.514464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:48.555192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ea3dbe56-5fbf-4626-9bbe-e7fb5df2cab6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058405 May 19 12:10:48.556469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 335/664] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:48.562134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:48.574879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2ddad45d-4ed5-464f-aeff-a4af87f464f2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.096317 May 19 12:10:48.575583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 330/665] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:48 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:48.977151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:48.978515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:48.984905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.985526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:48.986384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:48.986806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:49.121775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:49.130369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a4fcc5a3-9e14-4b7f-8cc7-a50c711bdba4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157579 May 19 12:10:49.130913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 336/666] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:49.568307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:49.569339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:49.576276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:49.576805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:49.577559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:49.577981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:49.593044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:49.593807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:49.600389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:49.600877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:49.601640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:49.602053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:49.608267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6bd74bc1-8202-44a3-a8f1-c1923afed261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043976 May 19 12:10:49.609110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 331/667] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:49.671918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:49.680119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-50047d76-c3c0-4437-b86e-2e5ebc3ffb03 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090899 May 19 12:10:49.680712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 337/668] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:49 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:50.147679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:50.148282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:50.154912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.155349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.156090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.156513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.292397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:50.300043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-406782a9-9fab-4f7b-b269-3e97d8d66827 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.156366 May 19 12:10:50.300700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 332/669] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:50.624524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:50.625919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:50.637175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.637888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.639139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.639853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.666834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:50.674359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2c59ae37-9a41-45d6-a391-e94ad5954066 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057009 May 19 12:10:50.674951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 338/670] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:50.697483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:50.698098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:50.705048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.705494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.706320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:50.706791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:50.771844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:50.779767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-955c4997-6e11-4c08-ab9b-81a781b3738f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086203 May 19 12:10:50.780350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 333/671] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:50 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:51.322674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:51.323657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:51.327352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:51.335492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.335932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.336699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.337123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.479764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:51.491262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-37afca5e-4725-4bef-9faa-8fc848549d11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.175729 May 19 12:10:51.492099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 339/672] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:51.689976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:51.691330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:51.697843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.698438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.699020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.699446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.730491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-645cd7a6-1f16-4d0b-b60b-e18b335b4388 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047335 May 19 12:10:51.731053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 334/673] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:51.800318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:51.800958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:51.808163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.808606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.809393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:51.809819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:51.872387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:51.880859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5503652d-ef15-4d2c-8fda-fbe94b317189 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086929 May 19 12:10:51.881431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 340/674] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:51 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:52.510392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:52.511003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:52.517512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.517949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.518700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.519125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.655023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:52.666926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7a4c57c1-b3eb-4731-8b02-5accad5218e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160077 May 19 12:10:52.667766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 335/675] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:52.742711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:52.743637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:52.750471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.750914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.751667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.752093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.782851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-85f08c20-bef7-49d9-a2de-e5d9ff21b252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044083 May 19 12:10:52.783315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 341/676] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:52.901057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:52.902058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:52.911869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.912307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.913100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:52.913523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:52.987802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:52.995636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-87cd45bd-a36d-4847-94a1-f104aa93b9dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.101107 May 19 12:10:52.996210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 336/677] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:52 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:53.691131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:53.691696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:53.699023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:53.699467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:53.700214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:53.700658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:53.796467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:53.797412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:53.804092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:53.804532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:53.805315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:53.805739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:53.838914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e938216-5e32-4c22-a527-8fac9b14840e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046387 May 19 12:10:53.839484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 337/678] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:53.845104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:53.852845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-721b5996-2494-4a05-9c06-4a69042841e4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.168109 May 19 12:10:53.853405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 342/679] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:54.016890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:54.017887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:54.028841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.029574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.030805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.031501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.047830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:54.100793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:54.108244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b30b3e32-8d58-4481-9b0f-84c7328681be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.097898 May 19 12:10:54.108839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 338/680] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:54 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:54.859258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:54.860800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:54.870508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:54.870905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.871110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:54.871349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.872201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.872975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.878003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.878503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.879539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:54.879967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:54.902104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-82fd5310-7b6a-4878-ace0-658f3d679c98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053955 May 19 12:10:54.902669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 343/681] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:55.013384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:55.021222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d298f41c-87df-4c18-bae7-88822d71e3b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.154663 May 19 12:10:55.021827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 339/682] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:55.129152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:55.130141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:55.136457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:55.136937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:55.137691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:55.138126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:55.213648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:55.221267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae8a2214-8ba7-4710-bb3b-5acacecf9be4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.099158 May 19 12:10:55.221847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 344/683] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:55 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:55.914567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:55.915865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:55.923967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:55.924485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:55.925407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:55.925924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:55.958356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b4df601-98e3-43d8-b3b7-1203077dd880 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048535 May 19 12:10:55.958918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 340/684] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:56.042521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:56.043488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:56.054417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.055147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:56.056374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.057137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:56.196519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:56.204076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a36fd5f7-cd0b-409d-8b04-c2aba937ef0c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.168211 May 19 12:10:56.204658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 345/685] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:56.238050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:56.238657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:56.245138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.245583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:56.246338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.246763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:56.309015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:56.316784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-301719ee-50b0-46f1-9161-e838583a51b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.082675 May 19 12:10:56.317619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 341/686] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:56 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:56.974688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:56.976277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:56.986968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.987719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:56.989028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:56.989748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:57.021028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c909b012-067d-4a08-a2a8-e86816ab148e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053063 May 19 12:10:57.021883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 346/687] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:57.224707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:57.225618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:57.236771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:57.237496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:57.238773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:57.239692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:57.343071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:57.343767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:57.350463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:57.350961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:57.351721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:57.352145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:57.388660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:57.396556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4c16d699-f5dc-485d-b916-389aed0da342 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.178478 May 19 12:10:57.397168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 342/688] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:57.422525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:57.434300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-90ee7d96-a2fa-43e7-88ee-f3ca30df890c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094925 May 19 12:10:57.435102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 347/689] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:57 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:58.037358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:58.038879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:58.049250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.049991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.051275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.051986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.083157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-483494c7-7380-4449-8a0f-c91989cf47a1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052593 May 19 12:10:58.083735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 343/690] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:58.422924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:58.423989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:58.435139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.435872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.437190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.437884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.451235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:58.451824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:58.458535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.458993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.459792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:58.460217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:58.522580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:58.530414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e7e4e246-e6b1-423c-86a7-61060333416c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.083045 May 19 12:10:58.530992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 344/691] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:58 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:58.575366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:58.583101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-59b4d889-1d12-420e-be29-3a176432fe8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166808 May 19 12:10:58.583669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 348/692] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:59.100029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:59.100963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:59.107942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.108403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.109235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.109678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.143186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-914d1345-f6f4-4db4-91c5-14d94e47f8cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050273 May 19 12:10:59.143738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 345/693] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:10:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:10:59.555576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:59.556615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:59.568008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.568804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.570202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.570692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.602238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:10:59.602811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:10:59.609975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.610404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.611164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:10:59.611587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:10:59.638127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:59.646053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a0291169-6b1c-4fe7-aa17-e3c35dc292d9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.097050 May 19 12:10:59.646796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 349/694] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:59 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:10:59.752735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:10:59.760961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5cb4cb9b-77b5-491a-b356-10939c763089 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162583 May 19 12:10:59.761572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 346/695] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:10:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:00.159848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:00.161400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:00.171558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.172290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.173755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.174464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.205072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a239d96-f70e-40e8-b398-d8d01e9cd37d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052242 May 19 12:11:00.205620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 350/696] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:00.667467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:00.668445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:00.679440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.679962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.680737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.681163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.753492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:00.761993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8858d62f-cf57-4da4-949e-26aa8e13eeb8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.101066 May 19 12:11:00.762465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 347/697] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:00 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:00.778355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:00.778959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:00.785822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.786262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.787015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:00.787431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:00.934663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:00.949480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-456c4250-f8da-4c30-807a-8874777e2e14 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174565 May 19 12:11:00.950244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 351/698] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:01.221403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:01.222880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:01.230737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.231184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:01.231934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.232366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:01.263034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-320af56c-f659-433d-b3c5-1ab6ecb4fa7b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048491 May 19 12:11:01.263598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 348/699] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:01.785498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:01.786163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:01.793569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.794018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:01.794777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.795217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:01.860706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:01.872511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-180fd779-92a9-4c95-b50a-167c112a96de tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091144 May 19 12:11:01.873088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 352/700] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:01 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:01.974457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:01.975480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:01.982885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.983327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:01.984078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:01.984505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:02.122741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:02.130592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-55f00c5d-3de7-4f93-b083-9014e0a50088 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162703 May 19 12:11:02.131168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 349/701] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:02.279741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:02.281262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:02.291803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:02.292563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:02.293860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:02.294568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:02.328836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8596c044-e5b9-4b97-af3b-b9e16f3c9745 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055888 May 19 12:11:02.329336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 353/702] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:02.893161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:02.894151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:02.904704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:02.905497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:02.906800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:02.907566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:02.977282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:02.985744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee99b731-638b-46d2-8b02-0d4981bf7785 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.098974 May 19 12:11:02.986336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 350/703] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:02 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:03.152436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:03.153423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:03.163946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:03.164703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:03.165996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:03.166710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:03.303607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:03.311259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-44de3faf-bd63-4952-bd9b-5fab14e34ca2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165542 May 19 12:11:03.311823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 354/704] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:03.341614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:03.342537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:03.349218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:03.349663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:03.350425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:03.350849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:03.381067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92fda51e-f9f5-4888-a668-ba0da3c2923d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043501 May 19 12:11:03.381637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 351/705] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:04.007091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:04.008075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:04.018632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.019369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.020616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.021379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.036543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:04.090924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:04.098544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c13af4a7-2975-4300-b281-afd0ba67425b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.098004 May 19 12:11:04.099081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 355/706] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:03 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:04.332697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:04.333637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:04.350457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.351199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.352434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.353175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.393766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:04.394694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:04.404199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.404987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.406190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:04.406896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:04.440180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e780af6-c4d4-48c7-8f91-5cd48b8a6407 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049915 May 19 12:11:04.440889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 356/707] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:04.513307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:04.521449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0801cf7b-91b8-419d-8c19-2b488e07b9f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.195218 May 19 12:11:04.522024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 352/708] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:05.116735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:05.117421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:05.125992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.126450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.127207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.127633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.191975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:05.203225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f98b80dd-f12b-493b-8d65-4742bd8a3593 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090322 May 19 12:11:05.203966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 357/709] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:05 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:05.456614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:05.457982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:05.465954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.466386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.467128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.467547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.499563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-71ced65c-8db8-4c75-bb4a-c991f7983898 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049232 May 19 12:11:05.500679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 353/710] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:05.538854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:05.539475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:05.546298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.546736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.547484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:05.547900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:05.681651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:05.689637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7ed3bdc1-9618-448e-a929-019018ce2f72 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.154788 May 19 12:11:05.690176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 358/711] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:06.223564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:06.224587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:06.234892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.235621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.236923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.237634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.308538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:06.316931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f0521e3a-94ef-4fc1-9799-0c6783d23b7a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.099883 May 19 12:11:06.317533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 354/712] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:06 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:06.512875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:06.513830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:06.520481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.520977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.521736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.522167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.554155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1969a841-fee9-498f-a632-ad7bf5fc549e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045317 May 19 12:11:06.554804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 359/713] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:06.710872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:06.711592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:06.720203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.720769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.721719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:06.722253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:06.725469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:06.878546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:06.889800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-48a73f11-6fe0-4953-84d2-f9c82b30bc0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.185266 May 19 12:11:06.890659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 355/714] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:07.338043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:07.339576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:07.349914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.351058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:07.351983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.352597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:07.413238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:07.421251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34f6d444-dd3a-4c0f-804d-5d25581f5a3f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.089721 May 19 12:11:07.421813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 360/715] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:07 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:07.570651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:07.572162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:07.575341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=78 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:07.584204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.584991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:07.585988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.586412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:07.616279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-de5ad778-2d3c-4662-a440-fb2c3bd1d6b2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052376 May 19 12:11:07.616823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 356/716] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:07.914168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:07.914726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:07.921890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.922327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:07.923078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:07.923500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:08.064452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:08.073793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6266beb4-085c-4519-beb4-c71291f5c5cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165917 May 19 12:11:08.074371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 361/717] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:08.439966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:08.440767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:08.449211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:08.449802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:08.450818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:08.451383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:08.515803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:08.523605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-69cc3126-beb1-4154-8860-afd83e8e41f0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088769 May 19 12:11:08.524237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 357/718] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:08 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:08.632771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:08.634028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:08.641124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:08.641616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:08.642447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:08.642921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:08.674334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afed593e-63b0-4c14-8e9e-1ad3683fb24c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048245 May 19 12:11:08.674892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 362/719] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:09.094710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:09.095702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:09.106882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.107504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.108789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.109481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.247741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:09.255911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a31fca86-da0b-4820-87bf-781a8ff83808 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.167686 May 19 12:11:09.256495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 358/720] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:09.541577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:09.542202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:09.549083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.549521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.550262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.550680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.611431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:09.619497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5c074cad-b338-4ad0-bb4a-40e992ae0304 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.081912 May 19 12:11:09.620076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 363/721] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:09 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:09.691028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:09.692443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:09.702723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.703547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.704859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:09.705586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:09.737861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a34490c3-5371-469a-a201-e27394b8e3c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053385 May 19 12:11:09.738614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 359/722] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:10.278796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:10.279824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:10.290299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.290736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.291483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.291906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.442785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:10.454015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f9a7ae8f-6868-463b-8eb4-aa8fc5772391 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.181670 May 19 12:11:10.454837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 364/723] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:10.646010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:10.646995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:10.654789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.655230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.655971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.656401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.717622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:10.725617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5d24425-38bf-4dd8-b1dc-bdb05b03ff1d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086071 May 19 12:11:10.726179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 360/724] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:10 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:10.750372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:10.751289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:10.758234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.758721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.759485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:10.759917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:10.789499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de43c648-a996-4472-b79c-8b55907a595f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043679 May 19 12:11:10.790099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 365/725] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:11.477643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:11.478554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:11.487884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.488554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.489778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.490435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.627735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:11.636078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-931ebc24-50d3-42d3-9e20-5cf3d38fc7ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164565 May 19 12:11:11.636623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 361/726] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:11.746930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:11.747812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:11.758748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.759336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.759945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.760372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.803026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:11.803937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:11.811019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.811465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.812230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:11.812668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:11.826347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:11.834562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6522f633-943f-4747-8a1d-8bcd719f976f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094942 May 19 12:11:11.835239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 366/727] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:11 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:11.847255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-80ada668-d101-4c2f-a3e6-1822ae474cec tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048167 May 19 12:11:11.847823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 362/728] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:12.657914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:12.658908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:12.669611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.670361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.671728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.672517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.809411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:12.821153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b75ae7b7-6a2f-465f-8b52-3d9fd2fe789b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170049 May 19 12:11:12.821892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 367/729] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:12.851665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:12.852718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:12.858462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:12.859295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.859477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:12.859757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.860508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.860960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.865665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.866111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.866871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:12.867294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:12.897239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d4918b89-e5e9-43ec-92d6-3b0cf6337324 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042601 May 19 12:11:12.898006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 368/730] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:12.925351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:12.933109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bce525f0-dfe7-4fd0-94c3-ada8331c01af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085296 May 19 12:11:12.933684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 363/731] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:12 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:13.841860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:13.842813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:13.853329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:13.853943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:13.856596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:13.857630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:13.909439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:13.910333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:13.917513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:13.918016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:13.918791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:13.919221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:14.014528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:14.022778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2a13b816-d58e-401b-9aea-e46b52222f6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.187356 May 19 12:11:14.023411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 369/732] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:14.028354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:14.029254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:14.036186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:14.036667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:14.037496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:14.037924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:14.102299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:14.110185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-30378887-100c-4119-b582-a7a7eacb1e32 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085810 May 19 12:11:14.110536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fe99f9d1-2277-47c6-8cd0-4d640cc45149 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.205231 May 19 12:11:14.110804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 370/733] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:14 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:14.114249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 364/734] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:15.045607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:15.046608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:15.058141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.058937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.060235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.061034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.122821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:15.123874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:15.130608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.131044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.131784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.132200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.161600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fa3f20ca-b3e0-4a0f-93ee-3cec6b4e92c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043263 May 19 12:11:15.162145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 365/735] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 12:11:15.166987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:15.167856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:15.174164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.174596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.175337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:15.175755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:15.204426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:15.212267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ceda5650-ef53-491d-8ea8-e740527de90e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174006 May 19 12:11:15.212937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 371/736] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:15.236151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:15.243938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e535d36-db31-4cd6-b97a-1e16fe8fac8b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080814 May 19 12:11:15.244498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 366/737] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:15 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:16.177702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:16.179213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:16.190421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.191131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.192389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.193142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.197096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:16.225120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2fe0d3be-39e7-48c3-a89a-43935712dc4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054034 May 19 12:11:16.225771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 372/738] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:16.229005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:16.229614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:16.236945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.237386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.238149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.238575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.261296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:16.261886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:16.268266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.268723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.269511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:16.269939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:16.335208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:16.343453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3250b6e5-0a8e-4aec-b761-b8ad9ed7ae13 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086446 May 19 12:11:16.343996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 373/739] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:16 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:16.376080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:16.383827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-390d3411-821d-4ba8-ae9e-382808773bde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.158963 May 19 12:11:16.384389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 367/740] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:16 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:17.241610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:17.242952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:17.250795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.251232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.251978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.252400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.282038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7b4d3904-e92f-48f2-809c-492438813d5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047295 May 19 12:11:17.282591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 374/741] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:17.359714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:17.360324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:17.367509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.367943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.368683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.369222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.400890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:17.401464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:17.407969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.408390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.409187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:17.409613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:17.433091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:17.440863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1157f2f1-3528-4da5-8096-54dfed72b933 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085069 May 19 12:11:17.441402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 368/742] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:17 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:17.545972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:17.557751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5a1228fa-962a-48c0-b74a-80e99fe0c188 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160563 May 19 12:11:17.558541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 375/743] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:18.300205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:18.301707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:18.312058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.312871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.314121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.314831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.357604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa59cefe-27e8-40b5-814d-2d89376dd948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064177 May 19 12:11:18.358181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 369/744] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:18.462132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:18.463110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:18.473702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.474474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.475691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.476354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.541745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:18.549282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-23027111-ac74-4d52-ad12-2b5f9f875734 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093771 May 19 12:11:18.549841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 376/745] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:18 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:18.581052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:18.581613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:18.588547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.589075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.589823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:18.590243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:18.726737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:18.738232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5affff10-59d0-4332-a773-aa7e23d41772 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160743 May 19 12:11:18.738992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 370/746] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:19.374683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:19.376165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:19.386359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.387091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:19.388335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.389090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:19.424359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1f831cad-becd-4b22-bc40-e6e21464ab0c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057036 May 19 12:11:19.424962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 377/747] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:19.570108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:19.571153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:19.581631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.582536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:19.583577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.584281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:19.652022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:19.654282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:19.660947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-663d6a64-3dab-4a4b-b041-ee432dfbee84 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.097362 May 19 12:11:19.661510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 371/748] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:19 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:19.762147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:19.762740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:19.769601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.770035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:19.770779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:19.771200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:20.127897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:20.139614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1724bb58-926f-4f24-93c7-b12fe24d6b59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.383084 May 19 12:11:20.140541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 378/749] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:20.441542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:20.442917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:20.451431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:20.451870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:20.452615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:20.453064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:20.490193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dcc07c4b-7abd-4bf5-98c6-c45d1982c09c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055968 May 19 12:11:20.490747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 372/750] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:20.681583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:20.682802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:20.693283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:20.693915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:20.695170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:20.695881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:20.756967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:20.764680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7c22c6a3-00b5-435c-94ff-a963445f1c4d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.089548 May 19 12:11:20.765254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 379/751] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:20 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:21.165883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:21.166834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:21.178021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.178788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.179991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.180728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.324889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:21.333006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28692511-54ee-4767-a760-e272df7bfe4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.173781 May 19 12:11:21.333589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 373/752] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:21.508116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:21.509549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:21.512488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=82 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:21.518210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.518651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.519410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.519827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.550191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-038aac53-8070-47a8-a5fe-421720a51abb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049091 May 19 12:11:21.550757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 380/753] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:21.785563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:21.786419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:21.796836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.797562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.798817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:21.799521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:21.864937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:21.873306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db3c74cd-981a-4c05-a7df-96924d7f1155 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094499 May 19 12:11:21.873879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 374/754] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:21 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:22.354299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:22.356220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:22.366447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.367175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.368470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.369209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.511474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:22.522612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bb7c4d11-a2ea-4276-b17b-3d9138ce036e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174763 May 19 12:11:22.523471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 381/755] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:22.562494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:22.563391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:22.570248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.570704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.571466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.571887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.607562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5c44b207-1de0-4a55-a7cf-19ee71257d4c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049082 May 19 12:11:22.608119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 375/756] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:22.893955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:22.894572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:22.904004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.904608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.905599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:22.906131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:22.972299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:22.980102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f8b50cf2-c73d-4072-9ee7-94951cb958ea tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092560 May 19 12:11:22.980667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 382/757] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:22 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:23.541268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:23.541944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:23.548475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:23.548935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:23.549709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:23.550131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:23.619730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:23.620675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:23.627656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:23.628118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:23.628902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:23.629323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:23.658350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c838133f-46f1-40f6-b3a1-19d7c68836e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042621 May 19 12:11:23.662220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 383/758] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 19 12:11:23.693019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:23.701764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2c1c63be-d20e-4cd2-8b22-d0ddf3c3f0ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164475 May 19 12:11:23.702332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 376/759] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:24.001367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:24.002134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:24.013410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.014059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.015231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.015808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.081794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:24.092316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-131374c3-b38a-4757-b16d-033f55ca018d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.097726 May 19 12:11:24.092916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 384/760] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:23 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:24.671782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:24.672798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:24.681907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.682638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.683395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.683854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.713685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-972a3034-62c5-4d26-b216-70c0a1469cb5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046253 May 19 12:11:24.714369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 377/761] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:24.722014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:24.722707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:24.733360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.733952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.735033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:24.735650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:24.880544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:24.888429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-90ac3e68-f7e0-427c-886d-65517090dfa0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172506 May 19 12:11:24.889037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 385/762] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:25.108785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:25.109407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:25.115819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.116274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:25.117062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.117488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:25.184857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:25.192717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90fc8f81-b767-4e94-b1d6-dc62071f8c18 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088065 May 19 12:11:25.193421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 378/763] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:25 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:25.727545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:25.728365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:25.735536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.735976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:25.736758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.737181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:25.768010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-93926699-0ca4-4c55-84bf-ad073bd69ef6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044386 May 19 12:11:25.768976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 386/764] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:25.910152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:25.911174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:25.920154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.920595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:25.921379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:25.921805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:26.057667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:26.065966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b0dfa489-73ba-4596-8dd4-fcbd81a04bb0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162399 May 19 12:11:26.066572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 379/765] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:26.213499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:26.214422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:26.222714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:26.223312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:26.224300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:26.224942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:26.292033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:26.300027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-572ba2f3-2acf-4366-9572-95185dddac42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092902 May 19 12:11:26.300612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 387/766] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:26 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:26.784071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:26.785629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:26.794533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:26.794973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:26.795740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:26.796131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:26.825633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9f367964-d0eb-439d-8b48-a4fd6ec0376e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048255 May 19 12:11:26.826201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 380/767] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:27.086667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:27.096042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:27.097320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.097777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.098696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.099124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.231188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:27.233422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:27.239686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9288f09a-9b85-429e-a76a-f7b9422cbc17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.159549 May 19 12:11:27.240262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 388/768] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:27.324369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:27.325046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:27.332164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.332661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.333500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.333952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.397636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:27.405332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11e64517-ef5f-41e7-a40a-966766b15075 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088486 May 19 12:11:27.405896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 381/769] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:27.841594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:27.842159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:27.849622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.850269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.851082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:27.851525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:27.882089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b14bd0e6-a034-4361-b3b7-2767622c63cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046588 May 19 12:11:27.882664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 389/770] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:28.257280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:28.257884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:28.264370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.264837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.265580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.266002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.402175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:28.410547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ce52a59c-70ae-4d1d-b5f0-52806ed288c0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157149 May 19 12:11:28.411129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 382/771] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:28.423207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:28.423849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:28.433247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.433692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.434447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.434899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.498265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:28.507133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e3cff4dd-0d52-4337-8bb2-d31a0b643d33 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088176 May 19 12:11:28.507133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 390/772] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:28.895249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:28.896167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:28.903106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.903543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.904287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:28.904702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:28.934694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eec6713e-cac6-4b1b-ac77-61e0cf5398a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043487 May 19 12:11:28.935242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 383/773] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:29.431477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:29.432524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:29.440217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.440734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.441572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.442039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.527433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:29.528035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:29.534789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.535205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.535968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.536385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.575119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:29.582880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d1af203c-6f5f-4b27-995d-aa49ed38a0f8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157925 May 19 12:11:29.583456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 391/774] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:29.597334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:29.605137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-563e93e5-7c06-48a9-9752-5240d09e62b6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.081582 May 19 12:11:29.605730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 384/775] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:29 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:11:29.947492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:29.948488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:29.955339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.955922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.956678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:29.957131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:29.987274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7c18da8a-93a4-4f90-b6d9-d5f349ed44c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044365 May 19 12:11:29.987904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 392/776] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:30.600393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:30.601042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:30.607936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:30.608376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:30.609168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:30.609597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:30.621763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:30.622537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:30.630273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:30.630693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:30.631484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:30.631914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:30.702130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:30.710999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1ce4a507-0d73-405e-8bf6-e7b13c126fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093611 May 19 12:11:30.711588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 393/777] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:30 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:30.761077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:30.769437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e4750df-fddd-4929-9b6a-c8e713cebeeb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172926 May 19 12:11:30.769905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 385/778] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:31.003883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:31.005422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:31.013655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.014094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.014872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.015253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.044476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5e2f1131-52e1-4b09-b16c-9b67afb7d7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047342 May 19 12:11:31.044995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 394/779] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:31.731740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:31.732505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:31.741723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.742308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.743306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.743877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.786614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:31.787364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:31.794173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.794617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.795370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:31.795794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:31.809845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:31.817790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11dba0f8-65ef-41b3-8143-7232becf33c5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092003 May 19 12:11:31.818367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 386/780] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:31 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:31.932401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:31.943850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9693752f-4586-484d-a152-1f5cfc7e71ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160918 May 19 12:11:31.944612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 395/781] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:11:32.059768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:32.060719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:32.067083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.067516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.068287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.068742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.101156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ba86df99-dedc-4ba8-a6ec-5a51a24f39d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046186 May 19 12:11:32.101707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 387/782] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:32.838998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:32.839681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:32.847652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.848165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.849091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.849585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.915507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:32.926789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f3ea71fb-4202-4861-8eb5-864e88a8f0f7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093889 May 19 12:11:32.927603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 396/783] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:32 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:32.964175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:32.965238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:32.972480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.972952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.973715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:32.974145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:32.989437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:33.114181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:33.115026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:33.120553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:33.122508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:33.122953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:33.123736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:33.124157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:33.128246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-74b5dffc-4989-456d-817e-48286bfe55b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170258 May 19 12:11:33.129031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 388/784] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:33.154455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7c85e8c7-2173-4304-a3ae-66407b7192d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044742 May 19 12:11:33.155009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 397/785] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:33.948114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:33.948760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:33.955350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:33.955857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:33.956622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:33.957081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:34.028917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:34.040166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a27d8d34-4624-4717-8cb8-23ed3c073182 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.097087 May 19 12:11:34.040921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 389/786] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:33 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:34.148097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:34.148768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:34.159143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:34.159827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:34.160825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:34.161335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:34.167445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:34.168429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:34.174800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:34.175450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:34.176223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:34.176706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:34.207251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f0f691b-d69e-47b8-87e3-163596273a51 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044250 May 19 12:11:34.207738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 390/787] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:34.298243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:34.307658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a84ceae7-d008-46f9-b2ff-deef44a81082 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165822 May 19 12:11:34.308201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 398/788] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:35.060241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:35.060861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:35.068290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.068799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.069598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.070045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.134862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:35.142547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63e7f514-27ca-4e22-8055-24d17530eb8d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087710 May 19 12:11:35.143114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 391/789] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:35 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:35.223172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:35.224596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:35.235632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.236362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.237664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.238369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.270874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a58e2ffc-b7f7-4308-ba21-242d19b7bb19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054399 May 19 12:11:35.271423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 399/790] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:35.329180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:35.330013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:35.339602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.340054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.340827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:35.341251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:35.474196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:35.483738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0bc4e8a9-6698-45ad-887f-5d07834aaef6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162189 May 19 12:11:35.484317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 392/791] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:36.159615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:36.160206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:36.168401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.169100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.170302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.170841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.243893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:36.252172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-45b50a14-e02e-45fe-ae84-5f4ddca704f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.096438 May 19 12:11:36.252741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 400/792] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:36 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:36.282918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:36.283834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:36.290757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.291226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.291987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.292402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.330666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ccf10203-022e-4e11-a93f-446499f52b0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051867 May 19 12:11:36.331219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 393/793] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:36.504759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:36.505697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:36.513766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.514204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.514954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:36.515404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:36.658725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:36.669921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-af610774-6a51-4c23-bfba-0e1ad4fa7273 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.171353 May 19 12:11:36.670681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 401/794] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:37.273680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:37.274467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:37.283848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.284781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.285614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.286305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.343022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:37.343944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:37.350897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.351337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.352096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.352545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.358865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:37.368076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e0f962cf-f45a-4cda-b2f4-428604511a64 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.101002 May 19 12:11:37.369137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 394/795] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:37 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:37.384757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-361311c3-d70f-4ed9-a2df-802c7ac77645 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045850 May 19 12:11:37.385355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 402/796] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:37.692564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:37.693350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:37.695464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=78 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:37.701229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.701664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.702409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:37.702825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:37.838236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:37.849880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2b536fbe-ffda-48b2-8db2-c89dd861ade6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161662 May 19 12:11:37.850712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 395/797] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:38.385434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:38.386039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:38.393076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.393526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:38.394284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.394734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:38.396801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:38.397921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:38.404796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.405255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:38.406005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.406412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:38.437719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9f9a4995-1e9c-4f09-aa3a-d40459ddd2f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044856 May 19 12:11:38.438365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 396/798] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:38.459544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:38.467356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96e3ab9f-2c71-48b2-a7ed-e6a23125dfe5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085811 May 19 12:11:38.467867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 403/799] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:38.871667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:38.872424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:38.879105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.879558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:38.880367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:38.880848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:39.014496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:39.026639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-24ea8fed-a798-452b-9231-6fe65bbc635e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161779 May 19 12:11:39.027265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 397/800] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:39.452134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:39.453195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:39.460762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:39.461254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:39.462079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:39.462547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:39.483631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:39.484201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:39.490846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:39.491281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:39.492032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:39.492454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:39.493267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e55f8ae0-f0e7-4663-bf0d-ea3671aa6137 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045599 May 19 12:11:39.493846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 404/801] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:39.558185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:39.566292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-af4dffee-b8eb-48b7-b8e7-35fa5c4c739f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086547 May 19 12:11:39.566995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 398/802] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:39 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:40.049738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:40.050339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:40.057178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.057618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.058368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.058791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.073601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:40.196989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:40.204912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-588fca5b-dbf2-400c-bbbc-f7656bdf051b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.159048 May 19 12:11:40.205489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 405/803] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:40.509548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:40.511520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:40.522651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.523385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.524631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.525402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.554529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f36cef7c-020f-4579-bf2f-27a970d1159c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051801 May 19 12:11:40.555114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 399/804] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:40.586708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:40.587283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:40.594363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.594801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.595549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:40.595970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:40.657096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:40.664751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58e75549-eaeb-43d1-87f9-6b3fe05129f6 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084410 May 19 12:11:40.665356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 406/805] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:40 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:41.226496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:41.227351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:41.237681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.238420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.239666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.240413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.395720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:41.404085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7786da59-8d46-4c36-9b34-98e7f126ad9b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.184229 May 19 12:11:41.406833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 400/806] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:41.571121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:41.572671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:41.583412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.584152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.585468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.586184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.616375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fdb24700-c25e-4670-bf35-78b53d037a24 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052102 May 19 12:11:41.617031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 407/807] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:41.684186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:41.684780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:41.693839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.694284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.695052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:41.695481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:41.758652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:41.766755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-41ccdbbd-0a50-416f-9b5b-eeb24edf9f01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087990 May 19 12:11:41.767259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 401/808] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:41 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:42.421788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:42.422395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:42.428854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.429301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.430052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.430479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.576033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:42.583952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d264bad5-f83c-4b7a-9863-da8dcb8d7077 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166129 May 19 12:11:42.584523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 408/809] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:42.628847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:42.629755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:42.636325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.636826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.637571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.637995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.667661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-38afb750-a1ed-436f-bfc0-4891b9848796 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042816 May 19 12:11:42.668203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 402/810] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:42.787778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:42.788625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:42.797097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.797535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.798279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:42.798701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:42.860513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:42.868134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-807e49f1-add1-41af-b6fb-a9c6407693e7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086958 May 19 12:11:42.868718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 409/811] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:42 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:43.616462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:43.617498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:43.628817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.629534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.630771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.631473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.680884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:43.681837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:43.688325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.688793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.689574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.690006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.720010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f1f88070-969d-4218-9ec0-de223b884feb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043491 May 19 12:11:43.720581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 410/812] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:43.775456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:43.783538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-44af220a-6e60-4eb2-9191-d78eebfdf3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.173661 May 19 12:11:43.784102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 403/813] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:43.888974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:43.889985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:43.900982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.901418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.902157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:43.902607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:43.966366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:43.977905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-be979904-a681-4742-b545-de406507f6e0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.095014 May 19 12:11:43.978641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 411/814] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:43 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:44.736080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:44.737548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:44.747721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:44.748166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:44.748948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:44.749366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:44.780385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-58b2e9f2-69b6-4a42-8f42-7e68006b070b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050920 May 19 12:11:44.780890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 404/815] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:44.801539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:44.802143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:44.808493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:44.808968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:44.809717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:44.810147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:44.949690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:44.957680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e82cf7ff-779b-4705-a802-4abde720a275 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160057 May 19 12:11:44.958218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 412/816] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:44.999178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:44.999783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:45.006666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.007113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:45.007850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.008280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:45.010915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:45.071486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:45.079913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b76d07d7-264c-47c1-bbd1-5e01dfaef505 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085097 May 19 12:11:45.080548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 405/817] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:44 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:45.792047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:45.792906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:45.802058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.802767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:45.803973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.804684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:45.840944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19072969-ce8b-46f3-9a3f-37005051fced tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053022 May 19 12:11:45.841496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 413/818] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:45.978440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:45.979420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:45.990371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.991097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:45.992364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:45.993089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:46.097158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:46.097767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:46.104755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:46.105203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:46.105961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:46.106389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:46.134534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:46.142296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ddbb3419-8af9-40e7-a8da-992499beb49f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170318 May 19 12:11:46.142905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 406/819] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:46.176658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:46.188495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-306bf746-c546-46f4-8d0f-fce708aa0a01 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093247 May 19 12:11:46.189137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 414/820] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:46 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:46.857586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:46.859081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:46.869954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:46.870793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:46.871550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:46.871973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:46.901565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f225f64d-0fd6-482a-a7e0-d5b57df508ef tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050851 May 19 12:11:46.902166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 407/821] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:47.163662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:47.164558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:47.172512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.172983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.173726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.174147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.205563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:47.206159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:47.213159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.213593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.214362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.214788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.280275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:47.288144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e863a2d1-9d5f-4daf-89c9-af483003eff8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086436 May 19 12:11:47.288745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 408/822] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:47 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:47.308556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:47.319395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-faa2d69c-1920-4401-b13a-6a925e6ff525 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162379 May 19 12:11:47.320260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 415/823] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:47.918225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:47.919679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:47.930130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.930862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.932106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:47.932903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:47.968604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-00c05596-25e8-45e0-8f79-ff3154b38ac1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057359 May 19 12:11:47.969195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 409/824] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:48.309117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:48.309815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:48.317419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.317936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:48.318825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.319324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:48.341564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:48.342225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:48.349055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.349495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:48.350233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.350655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:48.389034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:48.396698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28aed876-4d67-46e9-a876-e6f05b72c679 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094142 May 19 12:11:48.397305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 416/825] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:48 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:48.483292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:48.495111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7fa13eea-8b88-4ad3-9865-b78cf46a2e5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157326 May 19 12:11:48.495754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 410/826] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:11:48.984791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:48.986260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:48.996543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.997011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:48.997784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:48.998191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:49.027373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f4508bcf-a4e6-4eeb-b057-377812f5843e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049323 May 19 12:11:49.027892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 417/827] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:49.418031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:49.418805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:49.427842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:49.428390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:49.429357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:49.429849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:49.496248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:49.504337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b354f5ff-643b-4d3a-bdd8-e8a4417b658b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093080 May 19 12:11:49.504918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 411/828] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:49 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:49.512165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:49.512822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:49.519692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:49.520151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:49.520926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:49.521357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:49.656208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:49.664153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d475f3d7-3152-4fa6-ba00-ea8cecab8021 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.155821 May 19 12:11:49.664736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 418/829] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:50.043283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:50.044205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:50.050725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.051170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.051926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.052351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.081840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-59c6c149-6b99-4782-8336-1f1688420a1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045002 May 19 12:11:50.082391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 412/830] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:50.525873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:50.526887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:50.533883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.534325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.535075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.535496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.607159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:50.614862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9d284b05-b1c4-4c22-abc7-00423cc7acbf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.095677 May 19 12:11:50.615428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 419/831] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:50 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:50.683973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:50.684594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:50.691362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.691799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.692588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:50.693026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:50.832460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:50.844241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c68f8fe9-6d8f-433f-ace4-11af7400d6ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165742 May 19 12:11:50.845082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 413/832] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:51.098203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:51.099684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:51.108189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.108625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:51.109409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.109829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:51.140343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1bf969e0-539c-4027-b3e3-70cc4a8781e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049068 May 19 12:11:51.140907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 420/833] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:51.636297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:51.636946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:51.643430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.643872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:51.644615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.645063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:51.711896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:51.720219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f9ae7604-e141-4ff1-9bb7-51798cde7d05 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090413 May 19 12:11:51.720819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 414/834] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:51 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:51.865642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:51.866572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:51.870633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=82 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:51.879564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.880298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:51.881587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:51.882308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:52.030613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:52.038457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bf32199b-e660-43af-a62e-8d61b939759d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.179358 May 19 12:11:52.039044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 421/835] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:52.154501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:52.155417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:52.161963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:52.162396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:52.163147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:52.163561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:52.193122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-64598a63-4d6c-4ead-8b1f-5c3fade8f8c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043618 May 19 12:11:52.193673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 415/836] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:52.739663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:52.740473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:52.748909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:52.749346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:52.750108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:52.750533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:52.755822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:52.814307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:52.821932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-54e3afa6-c8ff-4ec6-8f03-3b06ee8398f3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088340 May 19 12:11:52.822505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 422/837] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:52 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:53.059465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:53.060500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:53.071531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.072157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.073593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.074023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.207225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:53.208157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:53.214922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.215239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:53.215400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.216110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.216530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.223302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5124919d-cd80-40fd-b709-086190a79f11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170358 May 19 12:11:53.223873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 416/838] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:53.246795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2c5fba83-5b53-41db-9c34-04e0441695fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043613 May 19 12:11:53.247349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 423/839] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:53.845612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:53.846392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:53.855347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.855813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.856557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:53.857009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:53.927071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:53.938892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6d461b19-14f8-431b-b6bb-2ee7e3c50ddf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.099368 May 19 12:11:53.939688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 417/840] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:53 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:54.245005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:54.245769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:54.253688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.254123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:54.254872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.255292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:54.259878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:54.260993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:54.267784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.268224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:54.269016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.269438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:54.303854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a0b68381-a317-4121-b033-bec4081a2950 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047933 May 19 12:11:54.304440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 418/841] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:54.388219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:54.397710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f7ccb47c-a6f4-4852-a48a-7fa70e52ab0f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.159711 May 19 12:11:54.398264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 424/842] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:54.962985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:54.964062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:54.970857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.971293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:54.972043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:54.972473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:55.036300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:55.047937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a0f6bd0e-f394-4896-a9cb-b8fab670e517 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091247 May 19 12:11:55.048724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 419/843] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:54 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:55.320247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:55.322069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:55.329958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:55.330431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:55.331200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:55.331618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:55.363955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0a3030e8-8b71-49c3-97b8-33813ad9fc6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050470 May 19 12:11:55.364526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 425/844] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:55.529013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:55.529615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:55.535623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:55.536058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:55.536830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:55.537250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:55.669911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:55.681232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d9758247-b970-4108-94c9-236f31e5eedd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.269909 May 19 12:11:55.682048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 420/845] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:55 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:56.068976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:56.069974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:56.080569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.081049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.081811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.083850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.149432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:56.157339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5d6aba24-2b9c-474a-898e-8ad8f6525851 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094912 May 19 12:11:56.157907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 426/846] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:56 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:56.380184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:56.381577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:56.390606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.391317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.392496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.393239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.420781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:56.427130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a9571aa5-e1ba-4371-bb21-706f628f2574 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054303 May 19 12:11:56.427742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 421/847] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:56.702890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:56.703703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:56.714479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.714928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.715669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:56.716102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:56.853879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:56.861615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9f8c2d10-afb3-41d2-a3a7-48681fe2f043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165350 May 19 12:11:56.862184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 427/848] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:57.181698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:57.182763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:57.189527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.189961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:57.190708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.191132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:57.260370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:57.268445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d2a3841-d0d9-4026-8752-47df63800001 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093277 May 19 12:11:57.269042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 422/849] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:57 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:57.439800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:57.440672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:57.447305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.447746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:57.448515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.448975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:57.478682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6bf08da8-45b6-4f72-82ca-a39c01ce97cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043045 May 19 12:11:57.479227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 428/850] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:57.883378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:57.884374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:57.892010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.892453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:57.893239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:57.893663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:58.261432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:58.272561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8eb08b40-47c0-42b5-adf4-949ccada192b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.395743 May 19 12:11:58.273391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 423/851] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:58.286155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:58.286783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:58.295145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:58.295592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:58.296350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:58.296815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:58.366618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:58.375163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-104d3747-b857-4178-96f1-c5c4b6dcb86f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093431 May 19 12:11:58.375799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 429/852] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:58 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:58.491979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:58.493000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:58.500944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:58.501442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:58.502359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:58.502856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:58.533266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ad1068eb-44d0-4e32-8fd2-661784079b1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046149 May 19 12:11:58.533795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 424/853] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:11:59.296046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:59.296726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:59.304953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.305471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.306359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.306885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.391568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:59.392186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:59.399270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.399725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.400471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.400925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.446545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:59.454321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4c6c1b2a-244e-4333-b854-c858eba34dba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.163508 May 19 12:11:59.454869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 430/854] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:59.463605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:11:59.471629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ff1afd91-f92c-4446-9c26-ede1f10988dc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084030 May 19 12:11:59.473119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 425/855] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:11:59 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:11:59.546425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:11:59.547281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:11:59.554146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.554617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.555378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:11:59.555804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:11:59.585269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9580ffcd-7c9f-41a2-a12f-3dedabcaf6e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043259 May 19 12:11:59.585802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 431/856] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:11:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:00.476042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:00.477029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:00.488197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.489219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.490438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.491185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.492553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:00.493982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:00.504171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.504171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.504171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.504171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.565656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:00.573440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f5841950-8f2c-4256-b8c7-94a24d4fc4d5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087214 May 19 12:12:00.573980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 432/857] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:00 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:00.597233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:00.598160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:00.604728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.605218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.605977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:00.606399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:00.629493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:00.636325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cf3f63f0-3daa-4a47-ae5d-7a738328f904 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043129 May 19 12:12:00.636940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 433/858] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:00.637643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-34fad0cd-d2d8-421c-9a58-b874cd602344 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.168685 May 19 12:12:00.638204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 426/859] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:01.594587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:01.595530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:01.605709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.606151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.606900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.607324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.647724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:01.648605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:01.655451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.655907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.656685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.657100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.670777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:01.678512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bdb77a08-3ca0-4949-ba27-8c7fad66116f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090710 May 19 12:12:01.679057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 434/860] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:01 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:01.683476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:01.684145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:01.686656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6b2f0430-3b71-4971-afba-1ff56bcbce13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042975 May 19 12:12:01.687232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 427/861] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:01.690140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.690583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.691332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:01.691757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:01.837933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:01.841601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aad53cbd-3379-4662-896b-461896db5e03 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161524 May 19 12:12:01.842194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 435/862] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:02.700001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:02.701269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:02.702747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:02.704283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:02.713045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.713859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:02.714637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.714778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.715058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:02.715213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:02.715970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.716398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:02.746233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-48d4bb00-58bc-424c-b229-ee9b5769ad1e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050319 May 19 12:12:02.746822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 436/863] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:02.777273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:02.785009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-47aa8fad-e33c-4c93-849d-2e5a0ecb0cbd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091953 May 19 12:12:02.785564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 428/864] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:02 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:02.862238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:02.863237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:02.874096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.874836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:02.876138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:02.876905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:03.022575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:03.035147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9e3219e1-db8d-406f-a6f5-08b643de34a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.179256 May 19 12:12:03.035918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 437/865] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:03.761580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:03.762583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:03.769491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:03.769927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:03.770678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:03.771100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:03.800808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-26c9b39c-f9f7-4418-93c0-f18c6e83f211 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045164 May 19 12:12:03.801373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 429/866] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:03.801932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:03.802749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:03.809754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:03.810179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:03.810941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:03.811369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:03.878267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:03.880470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:03.887517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eeebf517-d988-491b-852d-420e16ea7a29 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.089466 May 19 12:12:03.888098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 438/867] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:03 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:04.058165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:04.058901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:04.066242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.067631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.068126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.208356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:04.216369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f7058c13-c238-49f1-bba1-70d8c921c0c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165017 May 19 12:12:04.216963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 430/868] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:04.817167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:04.818639 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:04.829761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.830488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.831827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.832610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.865003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-166c9996-1767-4887-9876-e2c01c852371 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054549 May 19 12:12:04.865557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 439/869] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:04.904992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:04.905620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:04.912796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.913237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.913994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:04.914419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:04.980429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:04.988269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-50790f06-5c78-4fe1-b319-47488c47406a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087245 May 19 12:12:04.988876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 431/870] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:04 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:05.237557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:05.238433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:05.248791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:05.249530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:05.250801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:05.251549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:05.399513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:05.407981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-53018a12-2347-43e9-b95d-d6eb3d3777ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.177215 May 19 12:12:05.408551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 440/871] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:05.765529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:12:05.765864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:12:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:12:05.881700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:05.883183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:05.891416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:05.891861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:05.892623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:05.893077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:05.923545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35fc859c-d663-4e8a-841b-db29f7c04353 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048437 May 19 12:12:05.924192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 432/872] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:06.009565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:06.010491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:06.018877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.019528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.020165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.020593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.082594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:06.090859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-14c04d66-18e3-4207-8453-212e53d923be tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087870 May 19 12:12:06.091438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 441/873] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:06 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:06.429074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:06.430038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:06.438199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.438642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.439397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.439825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.581368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:06.589497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a0bb66a-3f92-4f65-9239-d13c8b80a363 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166966 May 19 12:12:06.590108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 433/874] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:06.942099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:06.943646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:06.954266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.955000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.956445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:06.957307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:06.999519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4aa86cde-ed52-47fb-9bd7-04619b1b608e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064665 May 19 12:12:07.000085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 442/875] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:07.107701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:07.108251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:07.115106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:07.115548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:07.116293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:07.116727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:07.186700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:07.194898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aff9699d-3b7a-419b-9548-51749fbefb49 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091364 May 19 12:12:07.195477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 434/876] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:07 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:07.610878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:07.611878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:07.622664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:07.623107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:07.623863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:07.624288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:07.761762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:07.774051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4208c592-08b0-4cd2-bb1c-362755a419b7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169461 May 19 12:12:07.774793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 443/877] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:08.015877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:08.017039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:08.019496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=80 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:08.026578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.027043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.027986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.028495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.061118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cec58f8c-4a02-4917-8f43-32120bb7a797 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051181 May 19 12:12:08.061703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 435/878] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:08.215704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:08.216740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:08.227104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.227836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.229126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.229832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.296462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:08.304284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9c8cb89e-21b5-4c9a-b8f8-afccc1aefd46 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.095034 May 19 12:12:08.304893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 444/879] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:08 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:08.795602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:08.796261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:08.803499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.804001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.804878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:08.805380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:08.952195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:08.965086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3f280e8d-060d-4630-bd38-4c205dc4ae86 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174695 May 19 12:12:08.965965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 436/880] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:09.077345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:09.078356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:09.085921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:09.086418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:09.087263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:09.087735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:09.119101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dedbbcaf-b59a-48c0-b2a0-efdb9f62009d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046909 May 19 12:12:09.119666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 445/881] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:09.325109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:09.326111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:09.337399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:09.338033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:09.339285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:09.340047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:09.403959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:09.412007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4229fac5-2058-4eeb-9619-7be420fbcc87 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093464 May 19 12:12:09.412582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 437/882] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:09 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:09.992267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:09.993686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:10.001873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.002315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.003040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.003484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.131660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:10.132668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:10.139243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.139753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.140571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.141098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.148527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:10.155373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:10.156717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0944218e-a9e3-4694-b3a3-651ae7607d35 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.171068 May 19 12:12:10.157270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 446/883] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:10.171835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f8de6cf3-4ae0-4c77-a5cb-f04700486738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044450 May 19 12:12:10.172576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 438/884] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:10.432877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:10.433963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:10.444264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.445052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.446279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:10.446983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:10.521138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:10.529386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3068c12c-bccc-495c-9931-a2227780f498 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.102911 May 19 12:12:10.529957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 447/885] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:10 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:11.178282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:11.179192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:11.187650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:11.189305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:11.190256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.190953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.192181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.192935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.199911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.200351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.201141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.201564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.231398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c252ee2f-870e-4319-a5a7-f17a90f2a7a4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050325 May 19 12:12:11.231976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 448/886] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:11.327738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:11.335642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-714ed7ff-824c-4d0d-96b5-243de7142881 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164096 May 19 12:12:11.336216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 439/887] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:11.546506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:11.547141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:11.553680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.554121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.554869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:11.555303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:11.634164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:11.642145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-abb1d7ac-0500-4cb1-aec4-785f272b3b71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.099619 May 19 12:12:11.642715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 449/888] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:11 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:12.244235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:12.245187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:12.251985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.252423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.253201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.253626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.282663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f942cbff-d56a-4944-8053-94402d1532b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042900 May 19 12:12:12.283206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 440/889] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:12.351956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:12.352589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:12.360161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.360782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.361723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.362397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.508930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:12.518471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e885321f-bfa4-4489-a47a-a2e8682d38f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.170385 May 19 12:12:12.519062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 450/890] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:12.767840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:12.768458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:12.774612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.775042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.775775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:12.776194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:12.840214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:12.848054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c95d799-89e3-43fb-8b87-d5dd537974c3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.191482 May 19 12:12:12.848611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 441/891] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:12 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:13.298170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:13.299641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:13.309929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.310670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.311917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.312708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.348473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eb157f85-2074-40e6-a1fe-851e8d65c1b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057187 May 19 12:12:13.349080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 451/892] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:13.534107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:12:13.534683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:12:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:12:13.535652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:13.536294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:13.543246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.543684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.544445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.544947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.686345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:13.694428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-232fbfde-5610-4a35-892e-2027e03d793f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162814 May 19 12:12:13.694994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 442/893] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:13.873969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:13.874720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:13.883210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.883682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.884620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:13.885191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:13.951310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:13.959482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dce25b61-c97a-4bd1-ae25-1959d9a0b963 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092129 May 19 12:12:13.960050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 452/894] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:13 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:14.031172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:12:14.031420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:12:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:12:14.364063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:14.365785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:14.375292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.375721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:14.376495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.376956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:14.406420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-894b8db1-f3ee-403e-8ba1-2275fb9406b7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049220 May 19 12:12:14.406978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 443/895] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:14.715828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:14.716862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:14.724863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.725306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:14.726089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.726545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:14.869645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:14.877552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c0583ba7-e62d-460a-8edb-4d90a6cc2fc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.167906 May 19 12:12:14.878127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 453/896] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:14.979482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:14.980341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:14.988872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.989311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:14.990063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:14.990487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:15.067462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:15.075839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-23c67931-c92b-4b44-b8f9-87d37c022c8c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.102968 May 19 12:12:15.076418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 444/897] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:14 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:15.423599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:15.424664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:15.432757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:15.433307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:15.434178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:15.434678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:15.467960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bcdf8e7f-b185-41b7-930d-dd508abd8766 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050443 May 19 12:12:15.468503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 454/898] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:15.903327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:15.904411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:15.911998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:15.912491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:15.913356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:15.913831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:16.053685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:16.061635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2aeb8c53-35c0-4f2c-b7ae-38f0e9ecc8fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164894 May 19 12:12:16.062253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 445/899] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:16.094656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:16.095283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:16.101895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:16.102339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:16.103117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:16.103540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:16.118618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:16.167693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:16.175901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ec0d0c6c-b067-4112-b7d6-358632a544c0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086501 May 19 12:12:16.176460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 455/900] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:16 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:16.484224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:16.485198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:16.491989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:16.492435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:16.493636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:16.494089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:16.526223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ff115f9-1129-4100-9487-2a9de28beab3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049057 May 19 12:12:16.526793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 446/901] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:17.082386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:17.082995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:17.090055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.090498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.091272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.091676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.196601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:17.197773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:17.206658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.207093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.207827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.208263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.231831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:17.239898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-565a58b7-f88d-488d-958f-35c373d5b219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.163358 May 19 12:12:17.240497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 456/902] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:17.270468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:17.278136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-25153f74-12d1-4439-ae5e-93e945ce7181 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088889 May 19 12:12:17.278699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 447/903] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:17 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:17.538734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:17.539658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:17.546345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.546794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.547557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:17.547975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:17.577857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e8ca4ac1-9627-496f-8c5e-46345cea0161 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043203 May 19 12:12:17.578414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 457/904] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:18.259084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:18.259719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:18.267482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.267981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.268960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.269493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.295537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:18.296178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:18.302644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.303086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.303834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.304478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.367761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:18.375410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-797dfe82-d5c0-4001-970f-c2ff6c543657 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084241 May 19 12:12:18.375981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 458/905] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:18 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:18.409701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:18.417920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0761d5df-ea91-4986-8dd1-86d1cafeb31e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.163822 May 19 12:12:18.418492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 448/906] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:18.596089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:18.597598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:18.607891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.608842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.609600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:18.610021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:18.639827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cbc733c-54a1-420a-90a5-e9c7947a81fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050768 May 19 12:12:18.640428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 459/907] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:19.395614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:19.396689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:19.412723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.413470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.414707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.415403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.439322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:19.440402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:19.451288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.452014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.453159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.453561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.491971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:19.501372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-26ce5305-7d62-4186-824a-4c1551fddae4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.111475 May 19 12:12:19.501372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 449/908] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:19 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:19.589258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:19.597013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-80d630c3-9b63-4149-b9d0-6ce1f3d6544b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164141 May 19 12:12:19.597579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 460/909] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:19.655856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:19.656823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:19.663225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.663659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.664405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:19.664844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:19.694209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d441c36d-7012-4050-809c-8097758f2798 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045283 May 19 12:12:19.694773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 450/910] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:20.520995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:20.522009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:20.532295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.532782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.533529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.533950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.609326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:20.614279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:20.614922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:20.617165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a31cf3b9-10fe-41b5-abbd-9d82d1adb8bf tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.102600 May 19 12:12:20.618021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 461/911] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:20 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:20.621875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.622347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.623100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.623527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.706237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:20.707151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:20.713959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.714247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.714981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:20.715404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:20.746038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-942183c1-36b6-46a9-82f7-bd8770f69987 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043797 May 19 12:12:20.746888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 462/912] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:20.759989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:20.768018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-de4a3697-b2be-4fef-8011-3a506f1b3219 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157519 May 19 12:12:20.768519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 451/913] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:21.638330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:21.639333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:21.646957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.647419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.648209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.648666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.717684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:21.725514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-276d18ca-8a52-4565-9314-7d96e24e6de8 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093807 May 19 12:12:21.726080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 463/914] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:21 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:21.758175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:21.759110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:21.766311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.766778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.767521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.767943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.785227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:21.785839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:21.792150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.792592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.793375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:21.793803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:21.798155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e1e4b47-8e25-460e-9bd3-9a7c9df1d4b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043991 May 19 12:12:21.798720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 452/915] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:21.934193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:21.946156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cef44ad-0a02-4da5-84bb-17dc6db92f63 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164676 May 19 12:12:21.946741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 464/916] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:22.743059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:22.743688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:22.750250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.750667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:22.751410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.751827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:22.810110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:22.811062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:22.813950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=88 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:22.815475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:22.819607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.820053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:22.820894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.821285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:22.823070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d064d261-4b43-4996-8603-7e26cf5d61fe tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084408 May 19 12:12:22.823877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 453/917] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:22 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:22.856517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-16f14323-df43-42c4-9764-583ec2787c64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050442 May 19 12:12:22.857111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 465/918] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:22.969911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:22.970899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:22.981115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.981839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:22.983059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:22.983767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:23.122203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:23.130001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-017d62de-df7b-471d-9f24-9efaa3af1913 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166626 May 19 12:12:23.130570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 454/919] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:23.843500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:23.844627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:23.855603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:23.856379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:23.857671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:23.858635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:23.867994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:23.869051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:23.876586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:23.877067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:23.877846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:23.878273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:23.881317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:23.914784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4530c358-56a5-4a25-958e-7215635d3e0e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051003 May 19 12:12:23.915383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 455/920] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:24.091165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:24.099148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d00cdfa3-1248-4746-bf6a-f560ff76c6a1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.262069 May 19 12:12:24.099635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 466/921] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:23 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:24.147283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:24.147883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:24.153944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:24.154384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:24.155122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:24.155582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:24.304223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:24.312048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f30afe18-9e59-4a3f-af8e-5741712e86fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.168751 May 19 12:12:24.312619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 456/922] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:24.928479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:24.929620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:24.937877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:24.938324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:24.939060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:24.939519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:24.968879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bb1bae39-957a-43a8-9073-799587da334a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045399 May 19 12:12:24.969414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 467/923] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:25.118525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:25.119254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:25.127118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.127562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:25.128313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.128761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:25.201834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:25.209736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-899cc373-8500-4732-af35-c9acbb56f43b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.096952 May 19 12:12:25.210309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 457/924] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:25 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:25.330939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:25.331572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:25.338365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.338808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:25.339559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.339983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:25.472105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:25.479827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-27ca3ca6-8708-47c5-8fdc-568da77aa335 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.153314 May 19 12:12:25.480260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 468/925] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:25.980975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:25.981886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:25.988538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.989026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:25.989807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:25.990378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:26.022338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2811d7bd-4cbc-4f57-9814-7f7206fc1a93 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045315 May 19 12:12:26.022920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 458/926] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:26.226822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:26.227430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:26.234303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:26.234751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:26.235494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:26.235914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:26.301732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:26.309318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-662dbc97-7577-4fa4-a7b3-4e4698ab558f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086490 May 19 12:12:26.309880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 469/927] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:26 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:26.497911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:26.498671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:26.507994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:26.507994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:26.508511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:26.508958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:26.646978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:26.659093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cbe354ed-aebe-4b7e-82b0-9721a0fb06fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165326 May 19 12:12:26.659831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 459/928] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:27.037731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:27.039663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:27.050024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.050771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.052022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.052742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.085548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d77f4d11-2b21-4398-8dfa-9a67799a323a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054486 May 19 12:12:27.086105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 470/929] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:27.329775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:27.330803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:27.339642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.340271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.341329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.341945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.404880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:27.412657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9431e98d-797b-4da2-811d-b79dc29a085e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.089177 May 19 12:12:27.413527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 460/930] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:27.684975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:27.685819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:27.696946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.697688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.698911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:27.699633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:27.836003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:27.844026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1fbe5fec-2bb5-4237-b02b-7546dd44966c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165779 May 19 12:12:27.844628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 471/931] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:28.098845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:28.099758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:28.106771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.107233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:28.107975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.108396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:28.140297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-51d4cc2a-d07d-4996-aff5-b43c642e70be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045672 May 19 12:12:28.140890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 461/932] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:28.433878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:28.434742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:28.445090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.445828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:28.447079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.447795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:28.452119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:28.512922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:28.520754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7cc9d977-90fd-4767-9527-f08cd8d28c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093511 May 19 12:12:28.521273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 472/933] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:28.869717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:28.870520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:28.878298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.878742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:28.879495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:28.879918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:29.021148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:29.032768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e13c455-127d-453f-86e3-c775a9c3d13d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169421 May 19 12:12:29.033602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 462/934] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:29.151693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:29.152609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:29.159551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:29.159987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:29.160838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:29.161258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:29.194271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-08d0a3a4-17f0-453f-81c0-86a21bd858e2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046554 May 19 12:12:29.194913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 473/935] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:29.539826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:29.540488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:29.547508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:29.547947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:29.548751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:29.549184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:29.612688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:29.624307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-13cc9014-a7eb-4391-ba16-a91c69b1823f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088462 May 19 12:12:29.624914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 463/936] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:29 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:30.059021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:30.060021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:30.067318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.067786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.068590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.069080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.207747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:30.208668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:30.209276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:30.215089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.215538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.216301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.216775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.217023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-303c1483-419a-474d-9941-09f6d5a2b4b0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164744 May 19 12:12:30.217611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 474/937] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:30.247885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4288af7b-066a-4b34-8b7f-ce4ccc343a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044132 May 19 12:12:30.248408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 464/938] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:30.641700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:30.642308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:30.648951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.649410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.650197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:30.650634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:30.718769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:30.726589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aabc790b-e545-4958-ba84-085049ff6799 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088953 May 19 12:12:30.727144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 475/939] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:30 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:31.237103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:31.237663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:31.244879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.245319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.246070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.246495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.259624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:31.260674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:31.267589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.268037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.268812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.269291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.298748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-72cc60c5-682e-4c83-b4fa-b6718ebcb261 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043146 May 19 12:12:31.299319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 476/940] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:31.389083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:31.397251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a6ac5be2-97f3-490b-88dc-27897d7204bb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165482 May 19 12:12:31.397847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 465/941] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:31.750218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:31.750858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:31.757686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.758156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.758958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:31.759412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:31.832582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:31.843874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8b6096af-9afa-4240-86a0-3861c71507e3 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.100116 May 19 12:12:31.844399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 477/942] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:31 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:32.313750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:32.314674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:32.321918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.322365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.323132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.323560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.355855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e36b2b4-c810-40fb-864f-3bb3450b6314 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048478 May 19 12:12:32.356436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 466/943] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:32.415373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:32.416078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:32.423020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.423464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.424228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.424667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.588424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:32.598714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9bff50f6-7a4c-444a-bec8-8eb6e654aace tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.187234 May 19 12:12:32.599299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 478/944] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:32.864235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:32.864844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:32.872664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.873246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.874094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:32.874572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:32.938852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:32.947005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b7722e3-8efb-419a-b896-bde5e4e35adb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088882 May 19 12:12:32.947550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 467/945] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:32 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:33.372478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:33.374175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:33.384536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.385354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:33.386747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.387445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:33.419274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0ba0bab8-6678-4b72-916a-f0ac83badf37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053973 May 19 12:12:33.419944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 479/946] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:33.619952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:33.621008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:33.632254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.634255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:33.635841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.636577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:33.787721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:33.799595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b6077cd-53fd-4df6-a598-b507160e0202 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.186287 May 19 12:12:33.800341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 468/947] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:33.972626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:33.973179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:33.980358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.980839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:33.981592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:33.982023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:34.043058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:34.051042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b599838-718b-414b-9148-0f4c948dffa9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084311 May 19 12:12:34.051602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 480/948] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:33 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:34.435519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:34.437034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:34.444567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:34.445036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:34.445798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:34.446222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:34.475813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7c532708-c752-4ed5-a3f5-f47197a11943 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047049 May 19 12:12:34.476353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 469/949] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:34.827641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:34.828683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:34.835771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:34.836226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:34.837016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:34.837444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:34.967344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:34.975008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eea330a0-82f2-4a74-aaf8-ab8f6dcdf5b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.154332 May 19 12:12:34.975565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 481/950] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:35.072021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:35.072898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:35.082575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:35.083016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:35.083835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:35.084194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:35.147449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:35.155345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-52d3ca27-21a3-499f-8623-b8fb7fdb92ec tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.089946 May 19 12:12:35.156002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 470/951] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:35 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:35.492193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:35.493586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:35.503889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:35.504339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:35.505140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:35.505559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:35.536666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dfc49f71-231e-4180-8563-e3d215f63cd7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051844 May 19 12:12:35.537220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 482/952] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:35.999770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:36.000797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:36.007750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.008190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.008974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.009400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.152028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:36.154754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:36.162345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-91d8300b-d0b7-42c5-9105-9182612fee0e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169086 May 19 12:12:36.162917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 471/953] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:36.171342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:36.171942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:36.178447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.178895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.179648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.180102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.243921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:36.251633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8889c6f4-87b4-4ac7-870c-112df0aef954 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084175 May 19 12:12:36.252192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 483/954] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:36 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:36.548406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:36.549349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:36.556221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.556665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.557427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:36.557851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:36.587587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bb7ca00d-1146-49aa-b0a9-bf59860abbc8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043237 May 19 12:12:36.588123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 472/955] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:37.183741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:37.184623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:37.195422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.196167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.197434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.198263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.271142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:37.271732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:37.279883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.280317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.281284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.281703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.341762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:37.348137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:37.349783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-98dbf36b-de7d-45bd-a3bc-e3a1e56b53ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172678 May 19 12:12:37.350360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 484/956] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:37.356631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5c5ad2ee-22a0-4cf5-8cc3-1022c763a9b4 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091133 May 19 12:12:37.357262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 473/957] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:37 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:37.576039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:37.576615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:37.584097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.584543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.585551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.585981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.599973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:37.601078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:37.607480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.607918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.608668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:37.609214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:37.646540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45a183a8-e99d-423f-b045-f040c8cd03a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050918 May 19 12:12:37.647109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 474/958] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:38.366533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:38.367200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:38.369479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=80 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:38.375211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.375709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.376434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.376874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.515165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:38.523438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fcba93e4-eea6-4922-8aee-98d2fd377189 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161250 May 19 12:12:38.524259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 475/959] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:38.528277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:38.528889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:38.534917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.535322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.536069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.536491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.609519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:38.618013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b55eb826-3f2e-40f1-84d3-d0938cf7bbed tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093099 May 19 12:12:38.618578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 476/960] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:38.660769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:38.661762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:38.668838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.669324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.670149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:38.670615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:38.703386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b1086271-6121-4917-8dd1-34ddc4d9c125 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048589 May 19 12:12:38.703935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 477/961] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:39.541160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:39.541809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:39.549300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.549768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.550560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.551016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.693052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:39.700825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-61a84b01-f809-45c7-b799-2d339e2e5220 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.163925 May 19 12:12:39.701393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 478/962] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:39.706016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:39.706615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:39.712808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.713251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.714004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.714429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.783139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:39.790891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd8594f7-07be-4104-b890-08909c2c5548 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088584 May 19 12:12:39.791469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 479/963] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:39 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:39.795987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:39.796910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:39.802703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.803139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.803888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:39.804315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:39.833181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e1b37c48-1973-4f9e-88bf-154b9a7eb709 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040695 May 19 12:12:39.833724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 480/964] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:40.721303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:40.722408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:40.733327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.734056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:40.735320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.736043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:40.885288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:40.893638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a95b9a81-b81d-4e0d-98c8-d78fa7e94942 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.178934 May 19 12:12:40.894223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 481/965] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:40.898807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:40.899444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:40.905798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.906238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:40.906992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.907414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:40.972628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:40.980665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-697ac1d0-a58b-4227-8254-9864e2b3b313 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085484 May 19 12:12:40.981276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 482/966] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:40 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:40.985817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:40.986708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:40.992786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.993224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:40.993971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:40.994386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:41.022783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-361f30c4-7932-4b14-9603-d822e2f3ad7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040539 May 19 12:12:41.023320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 483/967] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:41.915147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:41.916207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:41.926778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:41.927212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:41.927966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:41.928382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:42.071673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:42.080066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ac39bc8-0cde-4691-bcee-0e61476ea239 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.171270 May 19 12:12:42.085696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 484/968] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:12:42.085696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:42.086138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:42.092965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:42.093500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:42.094298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:42.094696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:42.158473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:42.166605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-97b6f779-f969-45ad-ba3d-fbe3e7632b9c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084938 May 19 12:12:42.167073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 485/969] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:42 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:42.171602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:42.172503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:42.178201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:42.178641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:42.179416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:42.179822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:42.210750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-345f6498-1007-4716-a50c-f2eb9fb732fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042581 May 19 12:12:42.211337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 486/970] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:43.100457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:43.101518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:43.112584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.113426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.114663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.115367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.261195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:43.269438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-afb5eadd-ac68-48bf-81c1-4ccf2ee086a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.175638 May 19 12:12:43.270009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 487/971] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:43.274361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:43.274938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:43.281149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.281592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.282342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.282766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.297848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:43.347532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:43.355657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-018b906b-e8ca-4f83-b451-9b24051b9b76 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084709 May 19 12:12:43.356226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 488/972] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:43 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:43.360797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:43.361697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:43.368102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.368541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.369445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:43.369872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:43.399862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5c2b0d8-a79f-4fcb-aba5-9e8938f56cf3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042627 May 19 12:12:43.400414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 489/973] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:44.290629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:44.291716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:44.301907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.302343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.303088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.303505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.442726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:44.450879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45c9551d-c215-4db2-b631-d7bbea8419ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166746 May 19 12:12:44.451450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 490/974] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:44.455819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:44.456379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:44.462178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.462616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.463362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.463814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.526851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:44.537673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b4868a64-f722-40e8-a216-fca035495cf2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085289 May 19 12:12:44.538248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 491/975] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:44 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:44.543097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:44.543937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:44.549994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.550438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.551199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:44.551630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:44.582656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-82f028c0-8456-48f2-b75e-31f14098aa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043404 May 19 12:12:44.583213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 492/976] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:45.472074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:45.473893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:45.480943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.481408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.482198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.482648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.618554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:45.626472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a1b3d7bd-f3c9-418a-ab1d-1b0049f42f56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160866 May 19 12:12:45.627055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 493/977] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:45.631488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:45.632080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:45.637920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.638360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.639111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.639532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.703245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:45.711418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b4dd0409-e35b-4d79-8d91-32c0fd3a7f53 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.083464 May 19 12:12:45.712020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 494/978] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:45 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:45.716538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:45.717498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:45.723688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.724065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.724861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:45.725272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:45.754411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0014faba-45ed-4a1c-85c5-1ad937bd4b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041399 May 19 12:12:45.754971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 495/979] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:46.647403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:46.648496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:46.659401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.660131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.661418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.662110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.808242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:46.816841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac2a727f-2792-4075-b012-e1a8b4b656fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.176011 May 19 12:12:46.817415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 496/980] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:46 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:46.822129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:46.822755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:46.828543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.829015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.829761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.830178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.891500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:46.899168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9fc050a3-54c8-40e9-a41c-1c4c815550b5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080735 May 19 12:12:46.899731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 497/981] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:46 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:46.904264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:46.905178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:46.911077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.911908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.912397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:46.912850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:46.943654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18baebaf-dcd6-4b32-9809-508eb84fb380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042929 May 19 12:12:46.944207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 498/982] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:47.581363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=41 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:47.604770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=81 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:47.838414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:47.839327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:47.849994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:47.850435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:47.851174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:47.851596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:47.996712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:48.007387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a8d3ef5e-3ab0-4b1b-934c-58f4744bde59 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.175378 May 19 12:12:48.008199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 499/983] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:48.013824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:48.014416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:48.022467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:48.022904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:48.023651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:48.024075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:48.095984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:48.107902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e6ed337-42fe-4f36-8134-678b15f8bf56 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.098301 May 19 12:12:48.108682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 500/984] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:48 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:48.115326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:48.116603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:48.122689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:48.123134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:48.123889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:48.124321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:48.156306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3be45ae-9ef5-41e3-832c-60d9a1ae5b21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046502 May 19 12:12:48.156893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 501/985] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:49.032212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:49.033526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:49.041448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.041883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.042630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.043052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.180483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:49.188611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-baf13604-7e95-434c-b02d-99e26536651a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162774 May 19 12:12:49.189203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 502/986] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:49.193589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:49.194177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:49.200255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.200684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.201462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.201883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.270389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:49.278507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-df0c8dc2-6cd9-4044-9dab-1a3b0f71a6af tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088366 May 19 12:12:49.279085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 503/987] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:49 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:49.283623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:49.284538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:49.290701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.291146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.291901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:49.292331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:49.321300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-08a5c9bc-926c-456d-9f06-1c966ade0b46 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041232 May 19 12:12:49.322002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 504/988] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:50.210281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:50.211424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:50.222843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.223548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.224915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.225648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.229914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:50.368305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:50.376437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ae4c7d82-d9a4-41dd-8165-ef7c9703fe78 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172877 May 19 12:12:50.377042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 505/989] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:50.381437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:50.382028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:50.388017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.388452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.389234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.389626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.455786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:50.465049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c2e54cf-a40e-481e-9df2-60743ee7503d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086964 May 19 12:12:50.465529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 506/990] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:50 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:50.470270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:50.471231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:50.477084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.477527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.478276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:50.478705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:50.508607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9f594711-9cce-445f-ad33-40cbe805418d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042031 May 19 12:12:50.509308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 507/991] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:51.398021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:51.399157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:51.409958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.410666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.411699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.412120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.563219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:51.571068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b8da7621-f421-4f03-ae07-6ff1c018497e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.179698 May 19 12:12:51.571639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 508/992] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:51.576124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:51.576738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:51.582674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.583118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.583870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.584298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.657369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:51.665601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-89a4ebba-8792-4479-a6c5-00592f2a082c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092899 May 19 12:12:51.666173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 509/993] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:51 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:51.670688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:51.671580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:51.677743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.678188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.678942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:51.679368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:51.709759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a9b17025-b464-4d3f-a5a8-39bb21ebe0cd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042561 May 19 12:12:51.710289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 510/994] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:52.591532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:52.592210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:52.601757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.602369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.603590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.604167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.746991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:52.757629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2c1f72f3-d37f-4591-bb6a-fbbeb45bcb9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172342 May 19 12:12:52.758202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 511/995] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:52.762730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:52.763251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:52.769287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.769733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.770518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.771008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.831935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:52.842027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a54becec-919a-4415-9bcf-14afef112e9f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.082540 May 19 12:12:52.842895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 512/996] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:52 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:52.847469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:52.848356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:52.854509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.854947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.856136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:52.856136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:52.885985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9839dbf0-dfc8-4b02-bf02-b6f836f4fba8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042073 May 19 12:12:52.886553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 513/997] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:53.779202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:53.780277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:53.790046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:53.790489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:53.791242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:53.791665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:53.937453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:53.949902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bf174b72-1218-4afd-a9e4-972ac9c64f70 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.177309 May 19 12:12:53.950652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 514/998] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:53.955024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:53.955603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:53.961652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:53.962096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:53.962850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:53.963278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:54.035320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:54.043273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8a6770b7-848c-4e00-9fda-0eed7c142f5e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091678 May 19 12:12:54.043835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 515/999] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:53 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:54.048590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:54.049521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:54.055473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:54.055895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:54.056655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:54.057109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:54.086995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b60fd6ea-48ea-4c80-8092-256ed737dacd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042161 May 19 12:12:54.087551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 516/1000] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:54.967827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:54.968465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:54.976185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:54.976800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:54.977710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:54.978143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:55.114945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:55.123280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bbe9f15b-e9aa-48a2-8f41-59972b45b049 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.159719 May 19 12:12:55.123855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 517/1001] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:55.128260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:55.128889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:55.134704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:55.135151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:55.135901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:55.136335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:55.197546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:55.205820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a3c3bbdd-38c5-409a-bbfc-5ec75f6af48d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080992 May 19 12:12:55.206391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 518/1002] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:55 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:55.211137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:55.212036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:55.218066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:55.218510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:55.219261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:55.219680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:55.249633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a8a9844e-040c-47d7-b7d0-704c49c6524e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042220 May 19 12:12:55.250200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 519/1003] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:56.144107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:56.145230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:56.153424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.153865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.154616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.155043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.303045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:56.311043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8085689c-53da-4e4a-a5b8-38d8689e6c1e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.173488 May 19 12:12:56.311611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 520/1004] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:56.316093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:56.316680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:56.322655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.323098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.323852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.324277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.390872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:56.393110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:56.399979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a20e28b6-9a86-4988-9563-dd5354723955 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087298 May 19 12:12:56.400479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 521/1005] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:56 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:56.405236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:56.406209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:56.412962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.413450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.414378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:56.414848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:56.443810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d37db7bb-93c4-4784-8a7f-b283b2639e04 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042310 May 19 12:12:56.444367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 522/1006] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:57.332326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:57.333386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:57.344738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.345514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.346762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.347713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.491227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:57.499472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-64996953-3ce6-4098-b843-f171f3141644 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.173810 May 19 12:12:57.500055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 523/1007] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:57.504531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:57.505154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:57.511325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.511762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.512508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.512967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.574107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:57.581716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4139ed29-7ce1-4cc4-9247-04b416dc503a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080698 May 19 12:12:57.582270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 524/1008] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:57 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:57.586898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:57.587722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:57.589344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:12:57.595281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.595716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.596458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:57.596900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:57.626232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1999e075-e4af-4a5d-853b-b081df59baa5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042992 May 19 12:12:57.626765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 525/1009] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:58.521141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:58.522132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:58.529708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.530142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.530886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.531306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.679128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:58.687098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e37fb41c-c075-48e2-8cf7-9dd5ca771043 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172598 May 19 12:12:58.687684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 526/1010] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:12:58.691964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:58.692549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:58.698356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.698794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.699533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.699957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.766032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:58.776164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ddf1115e-5bba-4fc5-92bb-3ce5f63cf655 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087264 May 19 12:12:58.776831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 527/1011] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:58 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:58.781362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:58.782250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:58.789517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.789927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.790699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:58.791123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:58.820397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a991b1d0-e5bf-4234-8c6e-436d986ef83f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042582 May 19 12:12:58.820961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 528/1012] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:12:59.708401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:59.709485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:59.720491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.721280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.722516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.723225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.862151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:59.871781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1a1ba851-969b-4cf1-a6c0-c5d2735f16fd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169845 May 19 12:12:59.872343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 529/1013] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:59.876665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:59.877268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:59.883706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.884147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.884953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.885384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.945615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:12:59.953535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4abe1f75-1fd8-4d98-a884-0dc073bee48a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080229 May 19 12:12:59.954104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 530/1014] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:12:59 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:12:59.958584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:12:59.959468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:12:59.965496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.965935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.966688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:12:59.967114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:12:59.997110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5b8e698-a887-4626-ac65-b658ef4afa15 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041981 May 19 12:12:59.997664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 531/1015] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:12:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:00.894058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:00.895143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:00.903492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:00.903933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:00.904672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:00.905138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:01.051704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:01.060075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0cc6b50d-f69c-4563-8e6f-0c645b6ac3ad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172573 May 19 12:13:01.060622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 532/1016] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:01.065262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:01.065856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:01.072079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:01.072520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:01.073311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:01.073739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:01.148967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:01.160469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a0eb356c-2c85-49b0-811f-0bf66eaf6175 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.098570 May 19 12:13:01.161148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 533/1017] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:01 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:01.165716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:01.166632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:01.172275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:01.172748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:01.173495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:01.173918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:01.203434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dd479d01-d509-41b2-a548-779c680f5941 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041270 May 19 12:13:01.203977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 534/1018] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:02.081937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:02.083008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:02.091489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.091936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.092817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.093256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.230322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:02.238221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90daefd2-ba0b-4a26-a942-9e833ee5a6aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.162957 May 19 12:13:02.238789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 535/1019] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:02.243499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:02.244089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:02.249769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.250209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.250930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.251373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.311898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:02.319838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb377ef0-fca6-4f63-bb44-c1b0643e846a tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080119 May 19 12:13:02.320406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 536/1020] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:02 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:02.324944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:02.325831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:02.331750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.332240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.332995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:02.333417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:02.363533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f28a9901-31f2-4fcd-a47a-3852863bd8a6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042028 May 19 12:13:02.364089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 537/1021] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:03.259729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:03.260548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:03.268884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.269431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.270352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.270899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.289468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:03.411300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:03.420193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-611cbd32-a4ca-48f9-8ad2-4fbbb1a1dc64 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166838 May 19 12:13:03.420760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 538/1022] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:03.425136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:03.425762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:03.431745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.432191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.432971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.433372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.496706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:03.506221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1a095153-1b31-4e12-8c72-7dfd63300017 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084425 May 19 12:13:03.516780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 539/1023] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:03 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:03.516780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:03.516780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:03.524454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.524911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.525658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:03.526081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:03.560625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e47c6594-8e33-4a81-ae82-99d196f3ebfb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052498 May 19 12:13:03.561214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 540/1024] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:04.441777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:04.442849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:04.454165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.454912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.456155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.456910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.592420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:04.600192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d2df9a5-8aba-4a82-8bf2-cc6898ea2ecf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165062 May 19 12:13:04.600789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 541/1025] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:13:04.605149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:04.605778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:04.612031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.612473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.613255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.613682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.681673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:04.689401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-05171bb2-3590-4d9a-b88c-ad11b059859d tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.087646 May 19 12:13:04.689997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 542/1026] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:04 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:04.694538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:04.695373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:04.701140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.701580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.702333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:04.702757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:04.732081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-42ea0db7-8068-4cdd-8e14-da53aad44338 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040901 May 19 12:13:04.732661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 543/1027] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:05.622200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:05.623325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:05.634171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.634611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.635365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.635789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.782036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:05.794018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c8923de-2b33-4bca-9b38-78c359ae353a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.178415 May 19 12:13:05.794723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 544/1028] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:05.800176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:05.801138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:05.809607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.810242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.811262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.811684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.880857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:05.889077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c7bfdf5b-8e38-468c-a337-8d46ad14bf42 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093349 May 19 12:13:05.889644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 545/1029] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:05 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:05.894442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:05.895345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:05.901642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.902080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.902823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:05.903242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:05.933266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e8b051b-b854-4f9a-91aa-d45a6b08ecaa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042637 May 19 12:13:05.933812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 546/1030] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:06.818207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:06.819220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:06.830569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:06.833617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:06.833617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:06.833617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:06.975719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:06.983671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-76d9496c-f5f7-4d67-8d6c-4a9ba034dddc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172262 May 19 12:13:06.984256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 547/1031] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:06.988587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:06.989208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:06.998070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:06.999568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:07.000335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:07.000771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:07.063087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:07.073799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bc5e83c5-192c-4bf0-83b3-e275aaa9d5eb tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088498 May 19 12:13:07.074299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 548/1032] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:06 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:07.078863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:07.079758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:07.085717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:07.086137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:07.086909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:07.087336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:07.119530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-494fa432-3ccf-4efe-9147-d809d1d0d1a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044125 May 19 12:13:07.120112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 549/1033] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:08.006138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:08.007207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:08.017822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.018539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.019766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.020475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.166124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:08.176586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-618f84fd-7561-462d-8c17-d54b61ec5132 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.177479 May 19 12:13:08.177393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 550/1034] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:08.182601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:08.183378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:08.189591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.190030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.190772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.191195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.254194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:08.263180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ab7786d-0753-4a19-9995-00b48b5b53ee tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.084560 May 19 12:13:08.273498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 551/1035] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:08 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:08.273498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:08.273498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:08.276306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.276806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.278235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:08.278437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:08.309335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1107a920-38cb-48e3-98d3-a07eb7466f64 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044636 May 19 12:13:08.309909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 552/1036] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:09.198394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:09.199152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:09.207193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.207711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.208593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.209132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.350068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:09.359330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a377e7e-bae4-4450-9267-59b7bbef2d01 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166667 May 19 12:13:09.359935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 553/1037] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:09.364872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:09.365472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:09.373785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.374204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.374964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.375376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.435703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:09.443532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c26c6e66-a40e-4214-aca1-7e4d6883c027 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.082658 May 19 12:13:09.444070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 554/1038] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:09 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:09.448693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:09.449583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:09.455354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.455789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.456530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:09.456981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:09.459286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:09.488420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c83955c-3344-435a-a352-08afbc3bdb84 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043307 May 19 12:13:09.489015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 555/1039] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:10.381250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:10.382366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:10.393889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.394633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.395887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.396604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.531350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:10.539275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-20ac69f6-2861-4816-afaa-3f3b4f87b01e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.164700 May 19 12:13:10.539871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 556/1040] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:10.544260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:10.544881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:10.550468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.550902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.551680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.552098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.615805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:10.624373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e533faf7-2f84-4de7-8f08-3ab22268f3f5 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.083573 May 19 12:13:10.624986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 557/1041] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:10 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:10.629473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:10.630364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:10.636166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.636541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.637312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:10.637733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:10.667555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c1f80567-83f6-44d4-8c36-9e6088b8a815 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041597 May 19 12:13:10.668118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 558/1042] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:11.558265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:11.559080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:11.568320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.568957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.569969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.570625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.724497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:11.732733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a0023e25-a2c6-4dfb-a824-f2cb870779c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.179867 May 19 12:13:11.733291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 559/1043] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:11.741106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:11.741106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:11.744796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.745359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.746145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.746587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.817427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:11.825532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8c9ee3b5-2290-4480-a7fb-7edabc7c0fbc tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091302 May 19 12:13:11.826110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 560/1044] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:11 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:11.830854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:11.831752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:11.837780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.838222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.838961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:11.839404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:11.869023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d34de58-fde0-4777-bcc1-bdc59392a51d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041665 May 19 12:13:11.869587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 561/1045] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:12.755010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:12.756175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:12.767260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:12.767885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:12.769161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:12.769865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:12.907530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:12.915164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-622fba90-33de-48ab-832d-38a020dab229 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166647 May 19 12:13:12.915790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 562/1046] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:12.920093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:12.920671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:12.926655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:12.926924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:12.927699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:12.928125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:12.994833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:13.002480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f4a5c88d-6a79-45d2-9247-432569088013 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085754 May 19 12:13:13.003046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 563/1047] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:12 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:13.007589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:13.008448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:13.014473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:13.014911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:13.015650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:13.016074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:13.044958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-20ad970c-054b-4c28-9547-325650dd2eba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040829 May 19 12:13:13.045431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 564/1048] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:13.939346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:13.940392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:13.951519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:13.951955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:13.952735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:13.953164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:14.090226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:14.098421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7c517b5b-3568-4589-a8cb-7930cce53f92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.168298 May 19 12:13:14.098987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 565/1049] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:14.103312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:14.103985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:14.109881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:14.110316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:14.111063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:14.111479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:14.173859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:14.186870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7b9c4971-102f-47b2-9453-a16f1d2d6ed9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086945 May 19 12:13:14.187509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 566/1050] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:14 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:14.192378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:14.193243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:14.199412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:14.199859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:14.200614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:14.201073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:14.230735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a2fa6ca3-400e-48da-bbf8-19f83d43ab87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042182 May 19 12:13:14.231291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 567/1051] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:15.116033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:15.116845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:15.123638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.124077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.124855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.125275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.270278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:15.278720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb0442b0-1e68-4a59-bb69-46fca09693f4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166984 May 19 12:13:15.279437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 568/1052] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:15.283925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:15.284513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:15.291282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.291722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.292477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.292934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.362593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:15.370937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-002facb4-87fa-46f2-ac48-ff077d7349e9 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090522 May 19 12:13:15.371513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 569/1053] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:15 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:15.376047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:15.376967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:15.382964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.383395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.384138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:15.384562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:15.414859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fef14afa-b414-44b9-a8ce-564ba2567278 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042365 May 19 12:13:15.415406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 570/1054] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:16.300631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:16.301808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:16.309544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.309976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.310723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.311141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.463069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:16.466421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:16.476135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c4ab5f80-9595-44e7-af25-dd641ba37e87 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.181776 May 19 12:13:16.477021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 571/1055] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:16 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:16.481317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:16.481901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:16.489484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.490086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.491245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.491973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.563948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:16.572015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-42ef74ca-a3dc-4d77-8acf-88882b96d4aa tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094044 May 19 12:13:16.572594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 572/1056] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:16 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:16.577174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:16.578083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:16.584156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.584603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.585373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:16.585839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:16.615236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ecdcfbbf-ba3b-42fe-87eb-906d282fe5b6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041633 May 19 12:13:16.615793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 573/1057] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:17.494468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:17.495180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:17.502706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.503150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.503907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.504334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.644271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:17.657044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b38c8d1-c179-4941-a7b2-2d9ca6c8d445 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166294 May 19 12:13:17.657904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 574/1058] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:17.662246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:17.662841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:17.666514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:17.673061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.673500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.674264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.674688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.735689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:17.744209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a292b422-ac0e-4651-8946-858c57e14576 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085370 May 19 12:13:17.744890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 575/1059] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:17 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:17.749301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:17.750198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:17.756031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.756504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.757344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:17.757771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:17.786753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-df0571b4-7263-43b3-b9e6-5eecdc84129f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040976 May 19 12:13:17.787341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 576/1060] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:18.678646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:18.679600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:18.687525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.688012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.688875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.689345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.831041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:18.839283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8404edb5-412e-46c7-a0a8-a38ab6eb0bce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.167215 May 19 12:13:18.839859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 577/1061] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:18.844376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:18.844994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:18.850801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.851233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.851984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.852407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.914472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:18.922119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-842565a7-bc23-4116-995f-2c05a62624d1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.081328 May 19 12:13:18.924088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 578/1062] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:18 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:18.928811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:18.929731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:18.935577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.936021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.936789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:18.937215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:18.967758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c7b0ea07-312c-4838-95be-2c35b6bbe2bd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042553 May 19 12:13:18.968322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 579/1063] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:19.860716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:19.861819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:19.873499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:19.874192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:19.875485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:19.876188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:20.018950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:20.028341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ca4bad6-a675-4089-9c30-5a9c7a08209b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.174166 May 19 12:13:20.028927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 580/1064] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:20.033435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:20.034056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:20.039791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:20.040336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:20.044141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:20.044141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:20.125148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:20.133138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2724f593-9944-43a8-9aea-c8d9f0b5c17c tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.103204 May 19 12:13:20.133710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 581/1065] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:20 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:20.138384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:20.139282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:20.145261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:20.145706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:20.146456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:20.146883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:20.176486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e422f02b-a384-4695-be9f-24148dbf23c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041783 May 19 12:13:20.177073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 582/1066] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:21.049716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:21.050698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:21.058950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.059387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.060162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.060550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.196664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:21.204384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-21effd7a-d3af-429f-a0fa-0fa2a0b4e75c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161219 May 19 12:13:21.204979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 583/1067] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:21.209444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:21.210110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:21.216136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.216582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.217375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.217805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.278292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:21.285887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-26dfa128-7f4a-43d0-8c74-a6284d216c7f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.079944 May 19 12:13:21.286460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 584/1068] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:21 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:21.291236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:21.292164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:21.297909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.298345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.299102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:21.299536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:21.328523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1d1706ea-0569-4fe3-b903-6c00bd542aba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041034 May 19 12:13:21.329123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 585/1069] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:22.226962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:22.227543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:22.234468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.234911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.235657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.236084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.374276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:22.385633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d1572999-377c-4be2-a7cd-4e6514e1b467 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166395 May 19 12:13:22.386385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 586/1070] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:22.390717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:22.391302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:22.398140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.398574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.399319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.399741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.461792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:22.469575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e23eede-3e33-4fc4-9af0-aca62aab3863 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.082196 May 19 12:13:22.470259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 587/1071] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:22 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:22.475021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:22.475968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:22.481897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.482344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.483099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:22.483522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:22.497945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:22.517769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c9cefbf9-df3d-4411-a69d-69aa1292c119 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046157 May 19 12:13:22.518351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 588/1072] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:23.407117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:23.408201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:23.417308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.417739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.418579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.418999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.569084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:23.579026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c14818e2-3160-4114-9bf1-e1cd57b6d354 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.178216 May 19 12:13:23.579616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 589/1073] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:23.583991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:23.584586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:23.591604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.592124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.593262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.593858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.666989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:23.674938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a5f955b5-6078-4ca3-ad5b-a66585c480ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.094381 May 19 12:13:23.675505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 590/1074] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:23 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:23.680045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:23.681055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:23.686760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.687259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.687990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:23.688438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:23.725833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c19d3364-8aab-4f50-b9b9-f89524de36b9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049242 May 19 12:13:23.726387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 591/1075] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:24.600193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:24.601312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:24.612390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.613524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.614792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.615507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.761301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:24.769027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eb4e0073-539a-45fd-b1d1-d88fdb16d539 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.175766 May 19 12:13:24.769501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 592/1076] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:24.773920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:24.774507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:24.781719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.782325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.783457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.784038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.852777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:24.860757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2b749a11-e435-4920-a400-0b55c79a6157 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090263 May 19 12:13:24.861333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 593/1077] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:24 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:24.866555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:24.867470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:24.873310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.873725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.874505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:24.874943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:24.905589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4ec8a2e5-305b-4d64-8090-71fabcfe643e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043260 May 19 12:13:24.906156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 594/1078] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:25.789445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:25.790214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:25.797958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:25.798481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:25.799372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:25.799864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:25.946165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:25.954979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c4c20986-0886-4a30-893f-4921e414a9cb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.172030 May 19 12:13:25.955582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 595/1079] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:25.959917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:25.960503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:25.966679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:25.967111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:25.967853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:25.968272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:26.029599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:26.037240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-384ee691-6bee-4414-a6f5-44230958c280 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080708 May 19 12:13:26.037809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 596/1080] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:25 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:26.042338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:26.043156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:26.048887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:26.049321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:26.050055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:26.050444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:26.078956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6bdf5db4-da12-48c1-a873-db3e3aacde8d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040186 May 19 12:13:26.079497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 597/1081] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:26.972919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:26.973625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:26.981361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:26.981849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:26.982691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:26.983173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:27.121568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:27.129471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-811a4ccd-57c6-496b-b75a-3738ea67a2f7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.161005 May 19 12:13:27.130045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 598/1082] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:27.134423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:27.135017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:27.140984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:27.141428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:27.142182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:27.142609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:27.203567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:27.211720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5f34c81-e879-4d8c-b9ab-81642047e838 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080694 May 19 12:13:27.212293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 599/1083] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:27.217500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:27.219439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:27.225825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:27.226340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:27.227091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:27.227563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:27.257828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c33466b7-c6d6-4a59-9963-6aa9e41a3638 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044518 May 19 12:13:27.258399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 600/1084] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:28.146529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:28.147168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:28.154207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.154649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.155433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.155883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.380901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:28.390695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2a765c2c-f8d5-40e7-93df-b6d540b2e74a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.248054 May 19 12:13:28.391285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 601/1085] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:28.395822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:28.396423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:28.403162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.403607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.404409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.404880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.473015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:28.480879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6073e6a5-a084-4e59-bc69-540d16681c0b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088597 May 19 12:13:28.481470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 602/1086] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:28.486352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:28.487216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:28.494026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.494474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.495226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:28.495657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:28.527052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b7f1018e-5d55-4d53-90f4-8be20ca10e7f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044548 May 19 12:13:28.527611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 603/1087] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:29.408152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:29.408933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:29.415540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.415980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.416750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.417174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.552916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:29.560909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5184d791-9848-4585-ba93-10cf735c7f95 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.156765 May 19 12:13:29.561489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 604/1088] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:13:29.565845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:29.566444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:29.572254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.572742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.573501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.573932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.577942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:29.634778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:29.642303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-beeedd64-5299-458a-aa65-63ad37d26fb1 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.079846 May 19 12:13:29.642880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 605/1089] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:29 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:29.647423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:29.648346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:29.654395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.654838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.655600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:29.656001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:29.685501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e80e1f0-6dfd-4fb6-939e-fdef9bd8a661 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041621 May 19 12:13:29.686065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 606/1090] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:30.582091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:30.583160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:30.590910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.591349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.592098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.592516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.734963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:30.743109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e941ff6-1ae6-479d-87cb-55df341cee62 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.167552 May 19 12:13:30.743674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 607/1091] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:30.748222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:30.748855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:30.754755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.755174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.755982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.756401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.819784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:30.827713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-03f8e1ee-d362-4cf8-9f01-669b338226ce tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.082993 May 19 12:13:30.828285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 608/1092] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:30 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:30.832938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:30.833835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:30.839821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.840264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.841051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:30.841474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:30.872016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-51456a75-b0c6-4908-9766-7e15c9282a65 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042645 May 19 12:13:30.872587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 609/1093] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:31.763915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:31.765041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:31.775932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:31.776712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:31.777960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:31.778671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:31.925109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:31.933654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0bbb96ef-1cff-413d-931e-fae3b5f36787 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.176732 May 19 12:13:31.934237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 610/1094] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:31.938600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:31.939191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:31.945286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:31.945728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:31.946479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:31.946903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:32.013447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:32.021392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-689e1fbe-ca77-4e96-9050-8937bcf664bd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.086172 May 19 12:13:32.021903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 611/1095] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:31 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:32.026646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:32.027545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:32.033498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:32.033930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:32.034676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:32.035099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:32.063347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b155a2b9-9dee-49f2-aca2-8932aa32676e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040475 May 19 12:13:32.063865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 612/1096] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:32.955475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:32.956475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:32.965078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:32.965517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:32.971581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:32.971581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:33.129588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:33.137537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-17207cbf-1fbe-4d2c-aa1d-44dd75e448b2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.188887 May 19 12:13:33.138114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 613/1097] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:33.145739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:33.146347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:33.153701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:33.154142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:33.154905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:33.155329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:33.227330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:33.239374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e7c1275-0e42-4c16-8e5a-45f850b78d21 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.100056 May 19 12:13:33.240626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 614/1098] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:33 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:33.247417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:33.248852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:33.257659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:33.258100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:33.258866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:33.259289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:33.291397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-16e25922-0c67-4857-8485-ea9f8056fe3a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050129 May 19 12:13:33.291981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 615/1099] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:34.158629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:34.159677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:34.167904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.168342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.169138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.169554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.310645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:34.318633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-72d071ec-89ae-47ed-98dd-3603aace9786 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.166603 May 19 12:13:34.319180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 616/1100] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:34.323473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:34.324059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:34.329730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.330169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.330905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.331327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.395801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:34.406200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ddadac85-cb2c-4f2e-b9d0-97c6842a7323 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.085692 May 19 12:13:34.406990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 617/1101] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:34 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:34.411648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:34.412465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:34.419839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.420428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.421632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:34.422241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:34.453969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-83bb9169-f6c0-4a12-93a9-bb4986708f76 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045991 May 19 12:13:34.454525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 618/1102] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:35.292485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:13:35.293127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:13:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:13:35.336042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:35.336677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:35.344157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.344604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.345394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.345817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.481191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:35.489340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-457c5e24-4621-4ebf-aa80-cb20a8192d27 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.157229 May 19 12:13:35.489913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 619/1103] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:35.494404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:35.494996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:35.501225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.501675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.502441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.502863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.564367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:35.572413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a9a23d40-7dd0-494f-8e95-99aa74a93243 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.081483 May 19 12:13:35.572998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 620/1104] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:35 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:35.577513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:35.578419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:35.584549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.585019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.585774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:35.586194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:35.612084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:35.617332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-121b97a1-f86f-456a-98b3-c5710b4ea4f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043366 May 19 12:13:35.617858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 621/1105] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:36.509086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:36.510053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:36.521311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.521873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.523026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.523446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.663997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:36.672240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9213650a-8367-4ee9-814c-cc6a7e0db9c2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.169544 May 19 12:13:36.672836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 622/1106] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:36.677258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:36.677846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:36.683816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.684256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.685032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.685453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.756268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:13:36.764383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5fb4e93f-257b-4352-ac5a-e0850d907d3e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.090544 May 19 12:13:36.764931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 623/1107] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:13:36 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:13:36.769466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:36.770363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:36.776450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.776925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.777672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:36.778096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:36.807751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-54018ae2-6d78-4ae8-b8b3-62676b3e335f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041834 May 19 12:13:36.808301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 624/1108] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:13:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:13:37.640919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:13:37.641983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:13:37.644706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:37.653725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:37.654457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:37.655670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:13:37.656424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:13:47.658323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:47.678799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=23 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:06.719207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:14:06.720215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:14:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:14:37.645519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:14:37.747247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:14:47.649684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=4 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:47.752139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:25.779331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:15:25.780436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:15:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:15:33.544234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:15:33.544757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:15:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:15:34.044885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:15:34.045175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:15:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:15:51.321174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:15:51.322438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:15:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:16:38.005961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-37b511c1-0776-488a-97fc-4c0a67c7361a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 202 len: 0 microversion: 2.1 time: 240.433880 May 19 12:16:38.006997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 485/1109] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:12:37 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 240436 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:16:38.012404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.013159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.021155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.021601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.022393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.022893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.082310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:16:38.084655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:16:38.086084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:16:38.086912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:16:38.088154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:16:38.089376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:16:38.090417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) May 19 12:16:38.091591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:16:38.094009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7328320-dfc6-4d8b-8293-3e6917b1e2b4) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:16:38.094009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d407e537-7700-4626-87fd-c4b1005aef92 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 180.452411 May 19 12:16:38.094009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 625/1110] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:13:37 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 269 bytes in 180454 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:16:38.094608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.095472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.102971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.103416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.104205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.104598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.164954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:16:38.173060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:16:38.173395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b65c95a-7aa1-478d-a26c-342b93ba962d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165350 May 19 12:16:38.174081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 486/1111] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:16:38.181617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14e1f1a1-83e0-4607-8838-841238a4387f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.093136 May 19 12:16:38.182144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 626/1112] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:16:38.186883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.187378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.199164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.199164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.199164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.199164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.262765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.263771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.269995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.270434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.271188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.271614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.300992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-25832a23-1f95-4743-a3ba-d429affba257 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.125487 May 19 12:16:38.301556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 487/1113] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:16:38.305888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.306479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.312048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.312519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.313404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.313825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.460435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:16:38.468409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bd18116b-800a-4b9c-8336-dc5c8bf2b3d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165953 May 19 12:16:38.469036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 488/1114] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:16:38.473351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.473944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.479556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.480000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.480787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.481216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.542215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:16:38.550051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0aba3f80-9acb-4d9a-b415-947fcd27e866 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.080046 May 19 12:16:38.550609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 489/1115] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:16:38.555058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.556012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.561746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.562189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.562964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.563388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.592939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-be7b3773-1384-4eae-be4d-f356543a4771 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041307 May 19 12:16:38.593517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 490/1116] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:16:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:16:38.598104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:16:38.598592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:16:38.604547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.605023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:38.605793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:16:38.606221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:16:48.193264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:48.215039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=10 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:48.603126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=12 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:48.624947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=35,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:55.302338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:16:55.302980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:16:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:17:26.732917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:17:26.734111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:17:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:18:38.352937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:18:38.396120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:18:45.793233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:18:45.794114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:18:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:18:48.357636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:48.401472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=4 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:53.554600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:18:53.555244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:18:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:18:54.057887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:18:54.058272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:18:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:19:11.334414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:19:11.335401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:19:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:20:15.311715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:20:15.312386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:20:38.579263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:20:38.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:20:38.583099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:20:38.584014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:20:38.585157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.586383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:20:38.587501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) May 19 12:20:38.588802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddc113a4-3fc2-443d-bc9c-409d03273fbc) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 240.399920 May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:38 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-c483e63b-3a8a-422e-ab32-1c5f27a6b4b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:20:38.590680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:20:38.591341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 627/1117] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:16:38 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 240402 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:20:38.591341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.591341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.597817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.598256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.599161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.599615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.621161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:20:38.625510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:20:38.626850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:20:38.627481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:20:38.628558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.629768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:20:38.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) May 19 12:20:38.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9d13770b-2772-40a1-81ff-e42f49a46d2b) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 240.032500 May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-704d2934-9aa4-4177-82e7-710d5abca438 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:20:38.633634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:20:38.634197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 491/1118] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:16:38 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 240034 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:20:38.638550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.639269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.645887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.646489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.647349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.647865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.711158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:20:38.719113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.088264 May 19 12:20:38.719638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (ip 10.222.0.35) !!! May 19 12:20:38.719638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (10.222.0.35) May 19 12:20:38.720214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-a225a8d7-e81d-4e4b-81f1-7322c1bc261b tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Unhandled error: OSError: write error May 19 12:20:38.720214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:20:38.720214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:20:38.720461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 492/1119] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:20:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:20:38.725099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.726011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.731974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.732465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.733210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.733633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.743321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:20:38.751073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.165728 May 19 12:20:38.751590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (ip 10.222.0.35) !!! May 19 12:20:38.751590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:38 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (10.222.0.35) May 19 12:20:38.752161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-1379de23-8e32-4be3-ace6-5ab350488dc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Unhandled error: OSError: write error May 19 12:20:38.752161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:20:38.752161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:20:38.752393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 628/1120] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:20:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:20:38.756922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.757410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.763038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041636 May 19 12:20:38.763451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.763629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (ip 10.222.0.35) !!! May 19 12:20:38.763629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:20:38 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (10.222.0.35) May 19 12:20:38.763907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.764096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-764adc2e-5ecb-4369-b816-cfe9dad13453 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Unhandled error: OSError: write error May 19 12:20:38.764096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:20:38.764096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:20:38.764415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 493/1121] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:20:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 0 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:20:38.764600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.765067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.768536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.769162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.774706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.775141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.775880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.776300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.914585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:20:38.925896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9445df61-6f75-4bbe-92ca-0b862401ddd4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.160461 May 19 12:20:38.926618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 494/1122] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:20:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:20:38.931268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:38.931855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:38.937665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.938101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:38.938847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:38.939266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:39.008221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:20:39.021046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4ddae242-5f56-49ca-9dec-f8881b727987 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.092720 May 19 12:20:39.021623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 495/1123] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:20:38 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:20:39.026356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:39.027263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:39.033249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:39.033695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:39.034453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:39.034881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:39.064294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96bc20f9-8ad6-4fd9-bf94-306e71dc01bc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041525 May 19 12:20:39.064868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 496/1124] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:20:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:20:39.224114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:20:39.224615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:20:39.231005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:39.231452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:39.232206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:20:39.232652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:20:46.747696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:20:46.748537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:20:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:20:48.760825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:48.783243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=10 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:49.229309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=12 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:49.250894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=34 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:05.806071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:22:05.806725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:22:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:22:13.564368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:22:13.565190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:22:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:22:14.070590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:22:14.070819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:22:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:22:31.347728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:22:31.348757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:22:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:22:38.962738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:22:39.067761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:22:48.967329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:49.072412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=4 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:35.320794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:23:35.322333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:23:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:24:06.761626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:24:06.762792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:24:39.179114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:24:39.181286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:24:39.182719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:24:39.183641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:24:39.185202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:24:39.186770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:24:39.188174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) May 19 12:24:39.189762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-53e10212-ac4b-45b8-af1d-d1d26573ce29) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 240.429458 May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:24:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:24:39 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-295c45fc-07f8-438c-8a7f-7211033ee72c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:24:39.191122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:24:39.191851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 629/1125] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:20:38 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 240431 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:24:39.191851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:24:39.191851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:24:39.202923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.203694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.204984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.205723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.298344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 202 len: 0 microversion: 2.1 time: 240.232174 May 19 12:24:39.299055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:24:39.299055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (10.222.0.35) May 19 12:24:39.299747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-f50a8804-cbe8-4a68-bb9d-8b2b2bb84e00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Unhandled error: OSError: write error May 19 12:24:39.299747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:24:39.299747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:24:39.300039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 497/1126] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:20:39 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 0 bytes in 240234 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) May 19 12:24:39.309580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:24:39.310611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:24:39.319337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.320089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.321217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.321836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.554247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:24:39.562811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.260568 May 19 12:24:39.565575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (ip 10.222.0.35) !!! May 19 12:24:39.565742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (10.222.0.35) May 19 12:24:39.566724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-715f8b70-7120-4644-a9eb-f2681dbe38c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Unhandled error: OSError: write error May 19 12:24:39.566724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:24:39.566724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:24:39.567140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 498/1127] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:24:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:24:39.575670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:24:39.576208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:24:39.584076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.584614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.585549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.586044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.652004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '19ccb0e5ef064cfda61c913034da54a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7ffaf0276394eafb5a4ee10ef3ab4d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:24:39.660261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 200 len: 1225 microversion: 2.1 time: 0.091871 May 19 12:24:39.660818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (ip 10.222.0.35) !!! May 19 12:24:39.660818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (10.222.0.35) May 19 12:24:39.661394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-ad817d20-41f9-4052-b1b7-573983637589 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Unhandled error: OSError: write error May 19 12:24:39.661394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:24:39.661394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:24:39.661646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 499/1128] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:24:39 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:24:39.667561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:24:39.668540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:24:39.674651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.675092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.675852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.676279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.707425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044849 May 19 12:24:39.708033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (ip 10.222.0.35) !!! May 19 12:24:39.708033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:24:39 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (10.222.0.35) May 19 12:24:39.708612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-87c80592-d0d2-4846-b526-a17e368343fa tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Unhandled error: OSError: write error May 19 12:24:39.708612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:24:39.708612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:24:39.708898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 500/1129] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:24:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 0 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:24:39.713241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:24:39.713722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:24:39.719327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.719772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:39.720516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:24:39.721033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:24:49.198889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:49.232436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:49.717735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=8 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:49.742211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:25.815853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:25:25.817001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:25:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:25:33.578339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:25:33.579196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:25:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:25:34.080203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:25:34.080521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:25:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:26:26.614925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:26.779227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:26.906015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:26.908296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:26.909786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:26.910674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:26.911790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.916369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:26.918498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) May 19 12:26:26.920316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-194ca0d5-f476-4419-9649-a5b563e68acc) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 107.724983 May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-9d10160b-34ec-4fa7-a16b-e39f930311c2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:26.921670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:26.922623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 630/1130] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:24:39 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 107729 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:26.922623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:26.923187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:26.944524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:26.947761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:26.950575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:26.953391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:26.955830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.958180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:26.960578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) May 19 12:26:26.963506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:26.965824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-70532716-e96d-4cc2-81fe-b16f484619d0) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:26.965942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:26.965942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:26.965942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:26.965942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 107.241373 May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (10.222.0.35) May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-28915ad9-cab7-4e16-ab61-a78c8c7c9ebf tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Unhandled error: OSError: write error May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:26.966370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 501/1131] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:24:39 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 0 bytes in 107243 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.044274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:27.110685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:27.112938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.114302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.114950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.117207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.118240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) May 19 12:26:27.119398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6dc5215c-e7e1-42c9-8b41-137a28427414) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 0.200085 May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-1f0b2033-ba29-4e9c-85a4-4a10698f0a8e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.121017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.121751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 631/1132] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:26 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 202 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.121751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.121751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.127156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.127567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.128327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.128769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.133680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.134501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.142171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.142612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.143000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.143363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.143782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:27.144556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.146205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.147155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.148832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.150390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.150390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.033489 May 19 12:26:27.150720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (ip 10.222.0.35) !!! May 19 12:26:27.150720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (10.222.0.35) May 19 12:26:27.151363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-b63402e7-a869-482d-8a3f-9ad9187d74dd tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Unhandled error: OSError: write error May 19 12:26:27.151363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.151363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.151625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 632/1133] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.156435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.157367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.163426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.163858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.164614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.165054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.196240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043637 May 19 12:26:27.196823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (ip 10.222.0.35) !!! May 19 12:26:27.196823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (10.222.0.35) May 19 12:26:27.197406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-0bbf5353-8b16-44eb-9a8c-90b1ad91fedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Unhandled error: OSError: write error May 19 12:26:27.197406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.197406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.197715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 633/1134] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 0 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.202183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.202665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.209144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.209580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.210332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.210752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.285597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:27.293903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.339597 May 19 12:26:27.294443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (ip 10.222.0.35) !!! May 19 12:26:27.294443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (10.222.0.35) May 19 12:26:27.295026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-061767b9-1656-4491-927b-738667c57eb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Unhandled error: OSError: write error May 19 12:26:27.295026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.295026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.295464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 502/1135] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.299779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.300263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.307183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.307924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.308739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.309277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.310609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:27.369642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:27.372003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:27.373760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.374679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.375861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.377235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.378273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) May 19 12:26:27.379386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-508dab56-d38c-4f39-a863-f6e3512a4716) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.175076 May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (10.222.0.35) May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-22e62c83-f3a7-45a4-b0ca-4509077ae5b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Unhandled error: OSError: write error May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.381824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.383032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 634/1136] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 0 bytes in 177 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.383032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.383032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.386801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.387284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.388069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.388495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.424759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:27.483946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:27.486123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.487545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.488240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.490498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.493192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) May 19 12:26:27.495646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af4acea4-f4bd-41db-9f28-72b1e6286c5b) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 0.191580 May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-8fb9b813-e044-44a6-9a76-9410b68186b8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.497863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.498708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 503/1137] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 193 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.498708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.498708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.510834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.511283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.512124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.512529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.526578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:27.528129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.529671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.530575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.531827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.039850 May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (ip 10.222.0.35) !!! May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (10.222.0.35) May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-d5ea0582-1747-47a6-a916-8d7f21effa71 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Unhandled error: OSError: write error May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.533024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 504/1138] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.533546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:27.535263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.536247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.540058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.164502 May 19 12:26:27.540796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (ip 10.222.0.35) !!! May 19 12:26:27.540796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (10.222.0.35) May 19 12:26:27.541385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-b048a0c1-fe85-49d6-879a-5537f280a1a7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Unhandled error: OSError: write error May 19 12:26:27.541385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.541385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.541615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 635/1139] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.542624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.543076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.543900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.544333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.546164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.546655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.552526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.553007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.553752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.554171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.573523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041819 May 19 12:26:27.574042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (ip 10.222.0.35) !!! May 19 12:26:27.574042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (10.222.0.35) May 19 12:26:27.574694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-243df427-b700-4dba-a169-533f1827ce90 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Unhandled error: OSError: write error May 19 12:26:27.574694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.574694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.574962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 505/1140] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 0 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.579334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.579821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.585660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.586114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.586873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.587301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.634945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:27.675491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:27.702036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:27.704207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:27.705607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.706467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.707544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.708839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.710418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) May 19 12:26:27.711737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f0ac4f1-4219-49be-aa12-2f05c8cd19b9) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.163259 May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (10.222.0.35) May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-d4a5c0a1-b913-45b9-abc7-7d5895462675 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Unhandled error: OSError: write error May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.713591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.714457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 636/1141] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 0 bytes in 165 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.714457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.714457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.717909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.718354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.719103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.719530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.737353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:27.739509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:27.741187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.742152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:27.743540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.745017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.746299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) May 19 12:26:27.747628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-066e7ddc-4bb2-477d-8c83-36e6911f7310) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 0.165203 May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (ip 10.222.0.35) !!! May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 (10.222.0.35) May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-9ae75230-0aaa-4412-87bc-80f756cabd0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Unhandled error: OSError: write error May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.749573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.750297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 506/1142] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 0 bytes in 167 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.750297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.750297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.753167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.753610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.754362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.754789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.768829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:27.770392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.771854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:27.772582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:27.773830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.028833 May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (ip 10.222.0.35) !!! May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab (10.222.0.35) May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-acd7b533-8a22-4f5e-b4da-bfdff4b37a2f tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Unhandled error: OSError: write error May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.774904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 507/1143] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.777438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.778411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.784419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.785128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.785683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.786087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.815213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041285 May 19 12:26:27.815733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (ip 10.222.0.35) !!! May 19 12:26:27.815733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments (10.222.0.35) May 19 12:26:27.816311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-b1993b46-688e-4619-9be1-82b15e4abd6e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Unhandled error: OSError: write error May 19 12:26:27.816311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:27.816311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:27.816586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 508/1144] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 0 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.821047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.821531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.827406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.827886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.828668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.829146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.855633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:27.864474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.156732 May 19 12:26:27.864980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (ip 10.222.0.35) !!! May 19 12:26:27.865234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e (10.222.0.35) May 19 12:26:27.865845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: CRITICAL nova [None req-d6f98594-152a-4018-923d-a3804faf5197 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Unhandled error: OSError: write error May 19 12:26:27.865845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova OSError: write error May 19 12:26:27.865845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova May 19 12:26:27.865845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 637/1145] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:27.870427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:27.870930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:27.876836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.877272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.877994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:27.879932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:27.940888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.000800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.031308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.033489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.034918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.035710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.036870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.038008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.039029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) May 19 12:26:28.040276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b8f19200-9710-46c8-aeda-268cff780daa) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.217525 May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (ip 10.222.0.35) !!! May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:26:28 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df (10.222.0.35) May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: CRITICAL nova [None req-350f2a73-6d72-4e35-b1f3-2b053885faff tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Unhandled error: OSError: write error May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova OSError: write error May 19 12:26:28.042736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova May 19 12:26:28.043442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 509/1146] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 0 bytes in 219 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 19 12:26:28.043442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.043442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.047944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.048388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.049240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.049685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.080370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:28.082803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:28.085551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.086793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.088981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.091855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.094055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) May 19 12:26:28.096629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.099145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-91cd679c-f0b9-4270-8c3c-7e6295e9f3e5) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.099145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14a5883c-b803-48c7-b019-6a6b861bf213 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 0.218145 May 19 12:26:28.099145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 638/1147] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:27 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 269 bytes in 220 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:28.099145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.099703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.104556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.105181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.105998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.106420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.120554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:28.122201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.123819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.124632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.125955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.127044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.127044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e697283-097b-46c6-b426-2a80dafa8ca7 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.036109 May 19 12:26:28.127044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 639/1148] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:28.129154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.130573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.137911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.138634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.139829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.140514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.173777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4e12a488-025c-458d-a08f-7be51952a96d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048640 May 19 12:26:28.174350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 640/1149] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:26:28.178985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.179460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.186163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.186608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.187368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.187791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.195587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:28.207871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cb9d2d40-d882-4482-b6cc-c854df5a9ece tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.170896 May 19 12:26:28.208459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 510/1150] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1561 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:28.213225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.213727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.219926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.220270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.221058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.221555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.270333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.323643 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'services' on Instance uuid 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.352702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:28.354845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.356600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.357521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.358654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.359782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.361833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) May 19 12:26:28.363481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:28.365198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d3cc900-b037-42cb-be40-21fc6fe31278) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.365198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92f71b24-20fb-4110-9ccb-30d89d020ba9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.181215 May 19 12:26:28.365198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 641/1151] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 269 bytes in 183 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:28.365198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.365832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.375986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.376796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.378136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.378901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.390788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 HTTP/1.1 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.394871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:26:28.397429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121', 'id': 'a01c6463-67e6-43b1-b807-547d04215062'} May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.398799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.400762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.402768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.405014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) May 19 12:26:28.407310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.408578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-331c6f2e-5abc-4272-b243-7a2f19e066fb) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.408578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01341314-4226-4391-a443-b55b66db14c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062" status: 400 len: 269 microversion: 2.1 time: 0.187569 May 19 12:26:28.408578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 511/1152] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments/a01c6463-67e6-43b1-b807-547d04215062 => generated 269 bytes in 189 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:28.408578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.409007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.416775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.417527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.418857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.419576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.436403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.438306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.439975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.440972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.442412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.443557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.443557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c73cbf86-737b-46ba-b327-5c3834a05361 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.040263 May 19 12:26:28.443557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 512/1153] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:28.445360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.446348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.452604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.453082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.453845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.454277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.484628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bd62f7e8-ee3d-4eb6-b769-48c7ebdc94dd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043054 May 19 12:26:28.485314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 513/1154] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:26:28.490403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.491039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.497238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.497603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.498366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.498791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.536307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:28.544124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dceb6390-0125-43df-9179-79b95bf3dc76 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.185436 May 19 12:26:28.544727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 642/1155] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1561 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:28.549418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.550358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.556387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.556869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.557622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.558046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.578884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.587432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c2cbf98-59ca-43ed-8443-c0867f29c386 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041526 May 19 12:26:28.587995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 643/1156] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:28.592504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.593119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.598933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.599354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.600091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.600504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.635506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.637672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.639262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.639928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.641009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.642144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.643170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) May 19 12:26:28.644272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.645541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f67fb78-b8bc-456f-8b67-c9d2878f45c7) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.645541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3c0b4cf2-38d1-4fa3-bda6-c15ec0311083 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.152682 May 19 12:26:28.645541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 514/1157] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 269 bytes in 154 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:28.645541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.646051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.650848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.651283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.652337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.652574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.666363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.667886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.670054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.671054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.672852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.674490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.674490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5c2d8304-43bd-42a9-9894-6df4d9139de2 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.028796 May 19 12:26:28.674490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 515/1158] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:28.674490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.675303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.681423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.681864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.682619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.683039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.713341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c94aec61-0184-4598-96c6-49b9d8e0993f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042486 May 19 12:26:28.713925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 516/1159] 10.222.0.35 () {60 vars in 1342 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:26:28.718615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.719091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.725691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.726168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.727002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.727432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.743630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:28.752209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f2574a91-ed32-4db7-80b0-a2743230c9af tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1561 microversion: 2.1 time: 0.162996 May 19 12:26:28.752691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 644/1160] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1561 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:28.757627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.758827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.765329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.765768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.766527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.766953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.797266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90721b58-f6f9-4d32-93b0-cfa78313136b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043487 May 19 12:26:28.798164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 645/1161] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:28.802500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.803217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.809258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.809708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.810473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.810900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.812032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.829380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:28.870951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.873625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:28.875196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.876474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:28.881344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.884590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.885871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) May 19 12:26:28.887253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.888529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b1405490-f0e0-4dc5-aea9-4b0f7befc185) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.888529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96d3444e-5db7-4208-8011-e184f41a0056 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.160208 May 19 12:26:28.888529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 517/1162] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 269 bytes in 161 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:28.888529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.888935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.888935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.888935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.889407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.889407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.909802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:28.911546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:28.913830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:28.915039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:28.916307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:28.916307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa8844cf-e491-4ae8-9cca-1df0fff1ddde tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.035973 May 19 12:26:28.916307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 518/1163] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:28.917991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:28.918503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:28.924475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.924975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.925692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:28.926113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:28.930406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:28.932541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:28.956264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:26:29.006333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:29.008516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:29.009965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:29.010630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:29.011860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.013116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.014719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) May 19 12:26:29.016406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.017924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-76455e7f-058e-4e4f-a156-4f6a17541d89) {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:29.017924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b7920868-ceae-4278-9b7c-14656ebdc41b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.211224 May 19 12:26:29.017924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 646/1164] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 269 bytes in 212 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:29.041115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38b11528-43cd-414a-bf00-255927c0def4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "DELETE /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 204 len: 0 microversion: 2.1 time: 0.125445 May 19 12:26:29.041732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 519/1165] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:26:28 2022] DELETE /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:26:29.046993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.048039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.054086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.054530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.055322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.055809 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.078070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-6a998255-d56d-4cd4-bf84-84771c16ba9a req-6b7f7ae2-389a-461c-a009-b99d2001c4b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:26:29.085677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6441e585-e391-4a73-b80e-026085fecd75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042723 May 19 12:26:29.086252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 520/1166] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:29.086863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-6a998255-d56d-4cd4-bf84-84771c16ba9a req-6b7f7ae2-389a-461c-a009-b99d2001c4b4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.087436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-6a998255-d56d-4cd4-bf84-84771c16ba9a req-6b7f7ae2-389a-461c-a009-b99d2001c4b4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.090927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.091414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.097688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.098168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.098899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.099320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.100761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-6a998255-d56d-4cd4-bf84-84771c16ba9a req-6b7f7ae2-389a-461c-a009-b99d2001c4b4 service nova] Creating event network-changed:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:26:29.108528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-6a998255-d56d-4cd4-bf84-84771c16ba9a req-6b7f7ae2-389a-461c-a009-b99d2001c4b4 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096537 May 19 12:26:29.109653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 647/1167] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:26:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:29.115336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.116018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.122934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.123469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.124311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.124848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.139681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:26:29.141442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.143020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:29.143819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.145111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:26:29.146658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:29.146658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b968c07d-62a0-4501-9976-f3c5cd9e5ca0 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.031960 May 19 12:26:29.146658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 648/1168] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:26:29 2022] DELETE /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:29.149026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.149508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.155770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.156211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.156999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.157420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.183638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:26:29.191315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lazy-loading 'services' on Instance uuid e2382eba-ee5f-4026-8253-b5835cfb84c4 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:26:29.233752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.compute.api [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Instance is already in deleting state, ignoring this request May 19 12:26:29.234572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c1df8d2-132a-4716-8b6e-bac22e5e5429 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "DELETE /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 204 len: 0 microversion: 2.1 time: 0.090280 May 19 12:26:29.235112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 649/1169] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:26:29 2022] DELETE /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:26:29.239760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.240696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.247305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.247662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.247789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.248488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.248938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Request method failure captured: May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df HTTP/1.1 May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLg7bw25grDiD414HpFXiib9i-KDiyveVzibvY00k_NQF6hT5XCTq7rLh7JSLy_PIxBJNwkzMw8dYeRODqDZjMz-xGg79xb7Spjjl6NtrhmLOyPbEUAk7Tx9xGLUNZ33Er6LZTiqjO1eNE7oy46NBoYx8nIx9VyuZ2kED5frYdsrpo May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:29.250040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_92c49142ea324049ba24b55ff84e5a3b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/92c49142ea324049ba24b55ff84e5a3b"}]}] May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 92c49142ea324049ba24b55ff84e5a3b May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1547752884 May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: d059ba56941148f3bb6080ba4f537f9d May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotTestJSON-1547752884-project May 19 12:26:29.251726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': 'e2382eba-ee5f-4026-8253-b5835cfb84c4', 'id': '08346d65-e7df-486d-baeb-57ee3caa61df'} May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:29.252484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:26:29.253618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.255079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.256759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) May 19 12:26:29.258344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.259557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5d2cda32-ceb0-42ac-96a5-37c8fe2d7832) {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:29.259557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ea6c350-ba00-49fb-83a7-a89650af9480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df" status: 400 len: 269 microversion: 2.1 time: 0.165531 May 19 12:26:29.259557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 521/1170] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:26:29 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4/os-volume_attachments/08346d65-e7df-486d-baeb-57ee3caa61df => generated 269 bytes in 166 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:26:29.259557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.259918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] [instance: a7b377f1-b40c-4146-8569-44510a0c33ab] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.265894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.266338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.267044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.267626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.281911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] HTTP exception thrown: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Request method failure captured: May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab HTTP/1.1 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjLnqHXbSQDM6yDHEDxeVauhaQvYSEy3mShaQxdpqcGNE8F9uttMfxj9sEL69zc3l9b2SJr0gsXIEuiFZhKwpZBYC81xI30aCFWtjkiqP-ctmShvh3BzyZ_MoWutk9XmqqFlXK3T5qAhPvQd2PXR6CRBBHrAjub0LdiFetB8KLudnW0 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:26:29.284224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b7ffaf0276394eafb5a4ee10ef3ab4d7"}]}] May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b7ffaf0276394eafb5a4ee10ef3ab4d7 May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestVolumeBootPattern-1824824684 May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 19ccb0e5ef064cfda61c913034da54a6 May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestVolumeBootPattern-1824824684-project May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.285791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a7b377f1-b40c-4146-8569-44510a0c33ab'} May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:26:29.286661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. May 19 12:26:29.288088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:26:29.290028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f8fe4e45-e781-4e83-97af-99b0943883e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048457 May 19 12:26:29.290028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 650/1171] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:29.291022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Returning 404 to user: Instance a7b377f1-b40c-4146-8569-44510a0c33ab could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:26:29.292165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8fac940e-469c-4425-8d5d-0b0ea3d7db7e tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab" status: 404 len: 111 microversion: 2.1 time: 0.037496 May 19 12:26:29.293349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:29.293570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 522/1172] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:29 2022] GET /compute/v2.1/servers/a7b377f1-b40c-4146-8569-44510a0c33ab => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:26:29.296768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:29.299957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.300393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.301188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:29.301612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:29.523765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:29.536911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-caf150ec-5f55-451d-b0db-b4597c0df804 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1567 microversion: 2.1 time: 0.245274 May 19 12:26:29.536911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 651/1173] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1567 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:30.097851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:30.098777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:30.106723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.107201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.107995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.108426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.139285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c93d61df-d99b-4eac-8e9a-94c955664db5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045375 May 19 12:26:30.139768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 523/1174] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:30.296616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:30.297556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:30.306965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.307416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.308198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.308604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.339831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ec49c200-6c6a-405c-8807-86db187d8666 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047161 May 19 12:26:30.340512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 652/1175] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:30.552542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:30.553619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:30.559539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.559970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.560747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:30.561168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:30.623312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:30.631730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c67b6baf-269f-4bf2-8b3e-3d2a2eee3ff3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1381 microversion: 2.1 time: 0.083053 May 19 12:26:30.632324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 524/1176] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1381 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:31.128611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-809abead-fc95-477d-97b6-0da32d108f5a req-c36a6ebd-aefa-46a8-8af0-15e7879abff3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08f420db-5800-43e3-9990-bd7aa43a9d3e", "name": "network-vif-deleted", "tag": "8eb1a804-edda-46f1-b9d8-6b463d0a7e0a"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:26:31.138666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-809abead-fc95-477d-97b6-0da32d108f5a req-c36a6ebd-aefa-46a8-8af0-15e7879abff3 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.139099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-809abead-fc95-477d-97b6-0da32d108f5a req-c36a6ebd-aefa-46a8-8af0-15e7879abff3 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.150905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-809abead-fc95-477d-97b6-0da32d108f5a req-c36a6ebd-aefa-46a8-8af0-15e7879abff3 service nova] Creating event network-vif-deleted:8eb1a804-edda-46f1-b9d8-6b463d0a7e0a for instance 08f420db-5800-43e3-9990-bd7aa43a9d3e on hv-cinder-80403 May 19 12:26:31.151778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:31.152921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:31.156211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-809abead-fc95-477d-97b6-0da32d108f5a req-c36a6ebd-aefa-46a8-8af0-15e7879abff3 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032231 May 19 12:26:31.156865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 653/1177] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:26:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:31.160020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.160465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.161257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.161667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.193721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8de4a52e-7596-46c9-9a07-3769daf2f686 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045824 May 19 12:26:31.194301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 525/1178] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:31.352848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:31.353816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:31.360513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.360979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.361735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.362153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.393961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c467522d-9aa6-478b-82f0-504112dc1c9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045778 May 19 12:26:31.394527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 654/1179] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:31.649420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:31.649933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:31.656546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.657034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.657806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:31.658239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:31.721213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:31.729611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-214ed96e-55ca-44ac-9a47-0b6b7d746249 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084337 May 19 12:26:31.730259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 526/1180] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:32.206108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:32.207131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:32.213509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.213946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.214673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.215126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.244916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2c9dba0f-a239-4f7f-ad71-047597925098 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042800 May 19 12:26:32.245475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 655/1181] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:32.406081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:32.407016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:32.414159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.414618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.415387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.415842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.448616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fbf22d97-c0f9-4db1-a8c6-cd2e8e1efbd2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046626 May 19 12:26:32.449438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 527/1182] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:32.752018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:32.752936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:32.760719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.761190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.762073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:32.762434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:32.830447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:32.838420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-91c406bd-a141-4b5f-a62c-4c64eb4e3ea7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091517 May 19 12:26:32.838989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 656/1183] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:33.261419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:33.263122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:33.273661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.274390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.275637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.276425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.308554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01f5579d-162b-4873-b279-505584013f8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054109 May 19 12:26:33.309539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 528/1184] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:33.461750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:33.462707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:33.469468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.469906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.470674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.471096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.506581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6ccba495-2b35-46fa-b3e7-8580b5acdc98 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048761 May 19 12:26:33.507174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 657/1185] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:33.855693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:33.856300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:33.863320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.863761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.864514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:33.864965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:33.932568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:33.940671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f1c3a077-a4e9-4afb-8bd7-19e8701e85e1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089012 May 19 12:26:33.941282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 529/1186] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:34.320711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:34.321626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:34.328579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.329057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:34.329813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.330231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:34.360573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be9747f1-3ca1-4660-b994-c3959050eada tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043810 May 19 12:26:34.361172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 658/1187] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:34.519793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:34.520737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:34.527283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.527748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:34.528482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.528947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:34.558210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3b451bcb-1bbc-4bb7-b5c3-d47c213fc482 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042384 May 19 12:26:34.558784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 530/1188] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:34.962189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:34.962816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:34.970401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.970851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:34.971615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:34.972037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:35.035762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:35.043685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-57dfcdf7-ebce-4e6e-bf23-ab3adbf27890 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085603 May 19 12:26:35.044260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 659/1189] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:35.373230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:35.374142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:35.381395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:35.381833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:35.382585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:35.383136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:35.412940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e36e85b-2a52-4371-8738-62568e4f0679 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043740 May 19 12:26:35.413501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 531/1190] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:35.570199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:35.571110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:35.578155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:35.578596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:35.579363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:35.579792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:35.611542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-164ba0df-415b-4f7e-96e4-09d0a816c1f0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045211 May 19 12:26:35.612018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 660/1191] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:36.060782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:36.061382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:36.067829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.068261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.069048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.069461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.133530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:36.144152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-41113f0e-c021-4981-9172-42ce2a54c904 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087085 May 19 12:26:36.144981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 532/1192] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:36.425254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:36.426171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:36.432833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.433246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.434112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.434541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.470812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1b211a32-6fe4-4687-9fab-9e07d3002408 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049591 May 19 12:26:36.472433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 661/1193] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:36.624403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:36.625345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:36.637694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.638428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.639680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:36.640434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:36.692326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c66b7f87-4dc4-454d-9c11-508fe2bbd5a0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071546 May 19 12:26:36.693270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 533/1194] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:37.162112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:37.162674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:37.169656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.170099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.170852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.171275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.235383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:37.243184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-197b38f4-824e-42ba-ad62-1f110db89fd2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084862 May 19 12:26:37.243840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 662/1195] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:37.483280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:37.484263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:37.490874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.491321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.492058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.492512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.510860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:37.526132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d67abf02-27b0-470a-b553-39d14181f0ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046847 May 19 12:26:37.526709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 534/1196] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:37.704597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:37.705453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:37.712091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.712532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.713481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:37.713907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:37.732626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:37.746955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0552c2f1-a2ec-4a9a-b3f0-060c162c5a77 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046438 May 19 12:26:37.747520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 663/1197] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:37 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:38.261074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:38.261737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:38.269194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.269645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.270458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.270864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.338370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:38.349313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a9c696dd-3abf-4856-aa13-6e28859e396a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092057 May 19 12:26:38.350035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 535/1198] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:38.540674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:38.541400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:38.547943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.548387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.549178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.549612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.580578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d21f2e4-80bd-49e6-b7ed-67ee87d226e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044191 May 19 12:26:38.581169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 664/1199] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:38.762871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:38.763786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:38.770518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.770959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.771723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:38.772149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:38.802188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a5d9b6b2-16e9-479c-8ef2-7fbf417c1955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045535 May 19 12:26:38.802644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 536/1200] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:39.370856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:39.371547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:39.378832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.379364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.380470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.380951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.450198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:39.461388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c47004ec-f881-4f70-8913-46c05188938e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097154 May 19 12:26:39.461984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 665/1201] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:39.592273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:39.593244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:39.600246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.600753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.601531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.601969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.642132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8119c717-f5a4-4b70-afc5-69a12ff62412 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054014 May 19 12:26:39.643054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 537/1202] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:39.816350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:39.817946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:39.826250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.826711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.827499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:39.827928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:39.859845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-50fb1446-3839-45db-8829-9ec6d9a9826f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049489 May 19 12:26:39.860416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 666/1203] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:40.484910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:40.485561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:40.492096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.492530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.493313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.493737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.556093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:40.564585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-688ea860-608b-4a38-90a1-a212e83a4430 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085450 May 19 12:26:40.565205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 538/1204] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:40.653644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:40.654548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:40.661808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.662268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.663035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.663469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.694052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b85429f3-10a5-45bb-8ea7-7daf9a61bd10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044519 May 19 12:26:40.694611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 667/1205] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:40 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:40.874259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:40.875162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:40.882320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.882772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.883528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:40.883954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:40.916406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8ea1e37b-6323-4049-b8dd-60c0ce5a9384 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047283 May 19 12:26:40.917025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 539/1206] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:41.582714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:41.583320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:41.590188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.590624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.591377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.591800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.660666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:41.670494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1fbcd7d2-ae8f-48e3-b2b3-e65714ec6066 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091688 May 19 12:26:41.670996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 668/1207] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:41.706078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:41.706996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:41.714152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.714831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.715697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.716235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.747146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1a942463-3cbf-41dc-bbdd-4fcad5f42467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045064 May 19 12:26:41.747726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 540/1208] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:41.928032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:41.928980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:41.936116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.936558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.937368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:41.937792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:41.971161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-88d741da-5fd4-4259-93f2-9f210cce4ed0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047203 May 19 12:26:41.973227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 669/1209] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:42.687534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:42.688148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:42.695477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.695924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:42.696756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.697186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:42.759858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:42.760726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:42.767796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.768224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:42.769056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.769459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:42.769658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:42.777611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c23c8e2f-6954-41df-b4dc-06969238589b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093936 May 19 12:26:42.778172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 541/1210] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:42.806832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f450bf66-903a-4be7-80c6-6afda7c5aff9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051011 May 19 12:26:42.807396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 670/1211] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:42.986691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:42.988153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:42.997640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.998079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:42.998827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:42.999247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:43.029094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-714580c2-7a97-4503-af9f-e6fd3bc472d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049015 May 19 12:26:43.029575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 542/1212] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:43.794663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:43.795853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:43.802985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:43.803438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:43.804192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:43.804591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:43.818793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:43.819707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:43.826356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:43.826847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:43.827602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:43.828030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:43.858243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-06dbbe00-07fa-4330-87e9-451bb308f728 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043319 May 19 12:26:43.859012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 543/1213] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:43.872376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:43.880575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f1b87bfa-dcbc-478e-9d21-fe9cf826bac7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089874 May 19 12:26:43.881174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 671/1214] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:44.040606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:44.041562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:44.047827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.048215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.048997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.049426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.078835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bf8886d7-a214-42ae-ba44-b5bd8b7ddb93 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042198 May 19 12:26:44.079381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 544/1215] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:44.874367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:44.875937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:44.887191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.887634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.888390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.888853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.897338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:44.898037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:44.904765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.905211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.905967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:44.906438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:44.919904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d4965e2-0d0d-48c8-a311-4d1fc6d3c821 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052386 May 19 12:26:44.920472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 672/1216] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:44.967929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:44.976251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-633f6f3e-f974-4e39-8397-4acd7c2db529 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.082720 May 19 12:26:44.976821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 545/1217] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:45.092081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:45.093168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:45.100743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:45.101258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:45.102131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:45.102625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:45.133087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0cac1938-747d-481b-8a03-a176e577ffcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045751 May 19 12:26:45.133560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 673/1218] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:45.934921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:45.935831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:45.945536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:45.945974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:45.946741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:45.947183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:45.979518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fc0e9622-c762-4856-8433-89e89b7080ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050663 May 19 12:26:45.980075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 546/1219] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:45.993376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:45.993951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:46.000677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:46.001148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:46.001886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:46.002308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:46.076966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:46.088712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7f3a11f4-d446-40f3-af5e-0d6bfce6af06 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098842 May 19 12:26:46.089550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 674/1220] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:46.144302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:46.145310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:46.152157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:46.152603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:46.153389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:46.153819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:46.183164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-984efc0d-9393-4091-bd08-c0edcaa2acf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042847 May 19 12:26:46.183749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 547/1221] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:46.995919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:46.997006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:47.005468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.005978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.006848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.007334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.052663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-29fa0a40-2a3c-4ae5-9567-ebd0a3c3262b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063638 May 19 12:26:47.053213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 675/1222] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:47.106103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:47.106708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:47.110102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:47.113813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.114245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.115012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.115432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.180087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:47.187924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-54153ec9-4451-4f0c-b8da-8f579b10672e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085830 May 19 12:26:47.188519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 548/1223] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:47.194983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:47.195891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:47.202497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.202939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.203689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:47.204112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:47.241015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77c38c04-91e2-4ea1-a31b-088065f5168d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049818 May 19 12:26:47.241543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 676/1224] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:48.070538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:48.072009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:48.079287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.079730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.080480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.080932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.110143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2127b59e-4a73-4650-96f5-9f91d41e7244 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046230 May 19 12:26:48.110771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 549/1225] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:48.208901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:48.209917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:48.218730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.219203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.219960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.220389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.253278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:48.254187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:48.261142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.261580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.262324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:48.262743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:48.283090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:48.290984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e8ca6d9f-9c32-4ddd-8306-7a8f29956edc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088653 May 19 12:26:48.291723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 677/1226] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:48.292632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-61f5464b-1a70-4e55-a8d8-9628f71d4519 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043637 May 19 12:26:48.293420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 550/1227] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:49.123718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:49.124673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:49.131763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.132209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.133007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.133435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.172109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5203f01-8217-4e34-95cc-4870cdc1a34e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053235 May 19 12:26:49.172701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 678/1228] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:49.312700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:49.313804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:49.314499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:49.314851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:49.318506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=97 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:49.325002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.325455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.326221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.326376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.326651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.326890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.327851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:49.327993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:49.330704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:49.357187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1d748120-832f-47f5-97c5-4fbb2dc5d646 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055468 May 19 12:26:49.357746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 551/1229] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:49.390745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:49.398590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb8ce265-51c2-4d9c-bfc7-539b46ee047e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092775 May 19 12:26:49.399159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 679/1230] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:50.188154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:50.189711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:50.199748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.200194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.200998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.201434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.232209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5235d2f8-5229-4d7b-aceb-c574e7701a81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050591 May 19 12:26:50.232755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 552/1231] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:50.371835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:50.372772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:50.380007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.380449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.381236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.381661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.385373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:50.412912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-af504faa-d733-469e-869c-7069894e312b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047140 May 19 12:26:50.413473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 680/1232] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:50.415769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:50.416396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:50.423286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.423753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.424513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:50.424977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:50.488167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:50.495775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19ea06ef-6a1b-4940-a8c8-22daf1893e1d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084200 May 19 12:26:50.496311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 553/1233] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:51.244960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:51.245904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:51.252433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.252903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.253639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.254062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.287965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3e65d941-54ad-44c6-9e3f-e00c2ba51458 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047406 May 19 12:26:51.288769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 681/1234] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:51.426088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:51.427177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:51.435112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.435560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.436333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.436796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.469342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1a455fa6-916d-49d0-b3b9-8593f5c7370e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048020 May 19 12:26:51.469885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 554/1235] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:51.513305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:51.513912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:51.520828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.521270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.521993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:51.522450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:51.590843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:51.598900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c660b51a-622e-409e-b296-926cdac34853 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089623 May 19 12:26:51.599434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 682/1236] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:52.300030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:52.301019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:52.307695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.308140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.308941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.309373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.340801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28315d41-43f2-4460-9a2c-8f49739a7558 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044749 May 19 12:26:52.341475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 555/1237] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:52.482790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:52.483743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:52.492183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.492616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.493403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.493829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.527850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0ef7498f-02d3-4ad2-8dde-dd78cbc3fe39 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049793 May 19 12:26:52.528528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 683/1238] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:52.619299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:52.620274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:52.627423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.627868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.628655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:52.629097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:52.702331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:52.710076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9c434985-2331-4a69-bf1c-93f50f6dabea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097485 May 19 12:26:52.710832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 556/1239] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:53.353651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:53.354694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:53.364191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.364669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.365425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.365848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.399369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0129a5c1-6d3b-4a4e-bbd1-831a73d63e95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049607 May 19 12:26:53.399914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 684/1240] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:53.539929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:53.540866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:53.547305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.547748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.548500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.548965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.578709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f4592fac-0cb2-4e15-9a74-6d6be7dcbd45 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042866 May 19 12:26:53.579271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 557/1241] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:53.731121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:53.732032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:53.743396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.744116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.745380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:53.746009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:53.816280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:53.824018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a57b9993-d083-4428-a1a5-18ef47ab1be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099350 May 19 12:26:53.824585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 685/1242] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:54.415583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:54.417108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:54.428236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.429033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.430266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.431024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.463052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-17baa12b-ef01-4d51-a68c-c0007523e543 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054099 May 19 12:26:54.463525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 558/1243] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:54.590702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:54.591605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:54.597985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.598433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.599181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.599614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.629779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-24c92753-beac-42fa-b0a6-a6937c160d05 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042749 May 19 12:26:54.630332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 686/1244] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:54.844888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:54.845907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:54.855868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.856320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.857109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:54.857536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:54.927116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:54.937581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-504eb769-578e-4107-9d91-67b188e65435 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099037 May 19 12:26:54.938241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 559/1245] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:55.336371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:26:55.337331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:26:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:26:55.477874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:55.479299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:55.486789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.487255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:55.487984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.488409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:55.519383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4fa06825-0d71-4ad4-8fe4-26554cde240e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048184 May 19 12:26:55.519971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 687/1246] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:55.644824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:55.645879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:55.654330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.654772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:55.655500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.655950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:55.685190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e7cbfef-8f51-466b-b150-c3b90a90fea2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046957 May 19 12:26:55.685744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 560/1247] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:55.956477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:55.957019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:55.963435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.963874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:55.964705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:55.965121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:56.035189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:56.043114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-080bf473-3d79-4474-b801-89fe7158c413 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091405 May 19 12:26:56.043654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 688/1248] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:55 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:56.532956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:56.533959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:56.540516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:56.540992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:56.541741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:56.542167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:56.572232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01aa79a4-ed58-45c2-a16d-e06a99c27edc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043309 May 19 12:26:56.572800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 561/1249] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:56.701066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:56.702581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:56.713641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:56.714443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:56.715176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:56.715601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:56.745205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7b140166-6667-4607-87e8-a3073f63d995 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050700 May 19 12:26:56.745817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 689/1250] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:57.065121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:57.066082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:57.076573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.077052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.077807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.078231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.148440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:57.156316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01202842-c91d-4cd6-a268-3e34720e99c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098889 May 19 12:26:57.156915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 562/1251] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:57.587567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:57.589109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:57.599299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.600026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.601317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.602057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.632355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-58432937-5b05-4a56-bc9d-bddea94cc2ae tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051391 May 19 12:26:57.632929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 690/1252] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:57.761669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:57.762695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:57.770517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.771067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.771998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:57.772528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:57.806331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9ebdbe46-13f6-4bfb-9a34-288ba107b53e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051815 May 19 12:26:57.806885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 563/1253] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:58.177290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:58.178230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:58.186934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.187376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.188121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.188546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.250607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:58.261886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-371aa4f2-9b65-457d-a4cc-dbc93b635093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091049 May 19 12:26:58.262455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 691/1254] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:58.648275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:58.649797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:58.657596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.658038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.658789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.659204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.688468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dcf028b9-dacc-43db-b296-6ac0ef7088e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046850 May 19 12:26:58.689060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 564/1255] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:58.823736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:58.825254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:58.835495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.836216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.837487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:58.838206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:58.867820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e15892bf-f08e-4bf3-ba44-f5f0e2cf6810 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050738 May 19 12:26:58.868366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 692/1256] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:59.287251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:59.288117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:59.296148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.296592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.297376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.297800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.377487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:26:59.388693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ee32cb90-1172-4a1b-89f6-1e1317a6c53a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.107871 May 19 12:26:59.389407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 565/1257] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:26:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:26:59.703791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:59.705309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:59.715588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.716029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.716826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.717254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.746543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-258d5377-a162-49da-bc98-5b32ae75a605 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049483 May 19 12:26:59.747091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 693/1258] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:26:59.884044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:26:59.885609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:26:59.892784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.893225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.893988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:26:59.894414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:26:59.924369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c73766c7-3880-47ab-8550-2916accb4e0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047416 May 19 12:26:59.924949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 566/1259] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:26:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:00.414141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:00.415136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:00.425714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.426493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.427704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.428445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.491475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:00.499271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c79be244-a4de-47f1-924b-7c2fb1cc703b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091567 May 19 12:27:00.500066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 694/1260] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:00.762529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:00.763472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:00.770538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.771025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.771849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.772313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.798974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:00.803912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2928ee42-1c15-4d86-ab73-a8d5aff81dfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047667 May 19 12:27:00.804467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 567/1261] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:00.940515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:00.942051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:00.953453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.954196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.955457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:00.956509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:00.989775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b26bef0b-72b4-417a-aaba-c0a8edfa74b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056190 May 19 12:27:00.990311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 695/1262] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:01.517671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:01.518296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:01.525174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:01.525622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:01.526403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:01.526834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:01.594429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:01.602206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3874b2b7-70c3-4e44-a082-4d1872387823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088766 May 19 12:27:01.602778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 568/1263] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:01.819914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:01.821440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:01.831925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:01.832721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:01.834000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:01.834864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:01.862104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:01.866856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-472dddbc-f8f9-4436-9b7d-9d5a21577711 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053868 May 19 12:27:01.867421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 696/1264] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:02.006326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:02.007846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:02.015504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.015946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.016689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.017148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.047766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-48ac4cae-d988-4916-8054-c6951be58a59 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048524 May 19 12:27:02.048308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 569/1265] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:02.623294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:02.624330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:02.636403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.637200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.638411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.639112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.709452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:02.717471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4cfcf383-dc00-43e8-bcf2-a47f3335cd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100868 May 19 12:27:02.718043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 697/1266] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:02.878320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:02.879587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:02.886791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.887239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.888002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:02.888436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:02.918002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ce1e5896-e3ce-42f1-b222-56288cae32dd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043710 May 19 12:27:02.918556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 570/1267] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:03.064120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:03.065644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:03.074802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.075252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.076011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.076437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.108674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b9c57d7f-a57f-4db8-8e98-45bb2ca82ba6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051927 May 19 12:27:03.109231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 698/1268] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:03.735825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:03.736465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:03.743977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.744499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.745425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.745926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.809164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:03.817197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2a0faf51-63d4-433c-9bd7-9e72bf9a1749 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086062 May 19 12:27:03.817772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 571/1269] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:03.933851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:03.934996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:03.943456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.944007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.945011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:03.945537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:03.979018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d60a8b5c-62a9-459d-9c1e-765fc989f34f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051657 May 19 12:27:03.979578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 699/1270] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:04.124998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:04.126486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:04.137800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:04.138602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:04.139950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:04.140726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:04.175020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-474b1ae1-10a3-4bfa-9838-cfcd4e7fbe24 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056857 May 19 12:27:04.175558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 572/1271] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:04.838293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:04.839350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:04.846356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:04.846915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:04.847738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:04.848204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:04.911069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:04.918664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e25eca01-c410-4842-87f8-98a53d302965 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086903 May 19 12:27:04.919225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 700/1272] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:04.994975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:04.996216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:05.003348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.003847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:05.005075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.005787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:05.035616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dccffb2a-8be9-48e0-8db5-5f40de7c0f4f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047243 May 19 12:27:05.036202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 573/1273] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:05.191084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:05.192141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:05.201326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.201768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:05.202515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.202946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:05.232739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c07dad2d-2f6e-48e1-a121-05171aa394da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048482 May 19 12:27:05.233293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 701/1274] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:05.936283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:05.936996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:05.944821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.945378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:05.946238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:05.946680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:06.010743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:06.018647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ad716546-d059-4bc4-b9f7-22600686b287 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086772 May 19 12:27:06.019205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 574/1275] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:06.046949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:06.047857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:06.054109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:06.054542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:06.055293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:06.055720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:06.085633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-40130ed5-7743-4da3-995f-2b82baea7809 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042666 May 19 12:27:06.086510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 702/1276] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:06.248770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:06.250146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:06.260908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:06.261638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:06.262876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:06.263583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:06.297755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-378c77e4-2ac4-4508-bd0e-6da66fba3664 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055850 May 19 12:27:06.298315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 575/1277] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:07.039186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:07.040087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:07.050874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.051601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.052882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.053591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.097863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:07.098687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:07.106199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.106645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.107412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.107835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.122210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:07.130250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-85e13663-a90b-494a-bb73-3c57d7ea4f28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097676 May 19 12:27:07.130799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 703/1278] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:07.140403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-df472991-951b-442e-98ac-4f67fa844832 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046514 May 19 12:27:07.141028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 576/1279] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:27:07.312985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:07.314206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:07.323677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.324145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.324931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:07.325359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:07.355774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d7491561-08c4-40d2-81db-506eb669f190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049449 May 19 12:27:07.356504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 704/1280] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:08.151344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:08.152687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:08.155371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:08.157141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:08.163593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.164347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.165635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.166346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.167053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.167684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.168436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.168899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.199685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa3b8f96-be4f-4e14-838e-42e76cb33a05 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050908 May 19 12:27:08.200352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 705/1281] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:08.229040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:08.236882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2f28d3ce-435d-4e9d-b34b-abc1498b9a2b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092210 May 19 12:27:08.237454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 577/1282] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:08.368374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:08.369363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:08.376239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.376662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.377438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:08.377860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:08.407820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cdf2770c-ae35-4970-82e6-64e61dd20190 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043370 May 19 12:27:08.408291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 706/1283] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:09.215487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:09.216561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:09.224700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.225203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.226191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.226734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.254255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:09.254890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:09.259716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b15ad8fa-56cb-4b11-88fc-4ef07b2ac911 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050684 May 19 12:27:09.260480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 578/1284] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:09.261547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.261996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.262750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.263183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.331101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:09.339475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86c8ae9d-daa6-4afa-accb-4d118e203c5c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089367 May 19 12:27:09.340053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 707/1285] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:09.423685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:09.425383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:09.434852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.435331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.436110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:09.436528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:09.468108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3cb4a5c0-5cc3-4e02-a088-87c92b1720aa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051069 May 19 12:27:09.468605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 579/1286] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:10.274937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:10.276414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:10.287833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.288578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.289872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.290588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.321692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bfc4b8eb-dddd-4899-80cf-6153743b8e2c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053452 May 19 12:27:10.322290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 708/1287] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:10.355443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:10.356107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:10.362956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.363395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.364140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.364562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.427878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:10.435914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-16543a7d-5e53-4df4-a7e8-2eab77a7f62b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084423 May 19 12:27:10.436472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 580/1288] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:10.479590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:10.480579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:10.487069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.487511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.488263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:10.488723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:10.520402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e663b07-33a7-4e01-97da-b7dc5eab116b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044794 May 19 12:27:10.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 709/1289] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:11.337850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:11.339406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:11.349841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.350476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.351714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.352419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.383591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-92a995fa-dd7e-4e89-99b7-d8daab18baf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052511 May 19 12:27:11.384235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 581/1290] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:11.456211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:11.456801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:11.464292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.464882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.465815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.466243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.532546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:11.533630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:11.534575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:11.540307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-55ff43b8-9207-424d-b8d5-a59dea0f1cf6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090144 May 19 12:27:11.540585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.540922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 710/1291] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:27:11.541107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.541945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:11.542480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:11.572311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7c9d7b96-20aa-4571-b419-ecad2806fcc9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044133 May 19 12:27:11.572883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 582/1292] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:12.401337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:12.402493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:12.409282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.409750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.410536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.410986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.441454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6a4ae6e5-35bd-40f1-8809-917cbc80069e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046750 May 19 12:27:12.441997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 711/1293] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:12.563322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:12.564314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:12.574650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.575088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.575834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.576263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.584086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:12.585069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:12.592495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.592979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.593735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:12.594165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:12.624366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-40f12ed2-038e-4ab0-af90-20a9f4447388 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044147 May 19 12:27:12.624966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 712/1294] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:12.637840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:12.648768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3269dcd9-2c02-463a-84e0-90d5d4bf76c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091626 May 19 12:27:12.649594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 583/1295] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:13.457495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:13.458967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:13.469288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.470025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.471241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.471974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.491792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:13.506088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cae326f9-9536-4071-b767-7bd417021348 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055225 May 19 12:27:13.506656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 713/1296] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:13.640118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:13.641792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:13.652481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.652953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.653734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.654142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.670624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:13.671193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:13.671296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:13.678093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.678535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.679291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:13.679724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:13.685097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8a2c97f6-cc9f-459b-9eeb-1bf8544238a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051534 May 19 12:27:13.685687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 584/1297] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:13.756001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:13.764294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-37bfb8ae-a881-42bf-bc8b-2a4b066d3565 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097626 May 19 12:27:13.764903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 714/1298] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:14.522030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:14.523512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:14.531189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.531633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.532388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.532879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.561750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fb4746de-6ed2-4305-afff-2fa146ff3e81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046360 May 19 12:27:14.562301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 585/1299] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:14.701044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:14.702513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:14.714118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.714566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.715341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.715762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.744930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fa3d48c7-69d8-4177-acbd-4c28cf24fc3e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050878 May 19 12:27:14.745479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 715/1300] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:14.785113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:14.785638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:14.794541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.794986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.795754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:14.796167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:14.864479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:14.872977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-681b5177-796a-426f-a397-9658cfb193f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094114 May 19 12:27:14.873704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 586/1301] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:15.579949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:15.581471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:15.591756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.592560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.593827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.594572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.630256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-10b73e8c-98a0-4a7e-9131-fac042e86800 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057131 May 19 12:27:15.630864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 716/1302] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:15.761200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:15.762044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:15.768668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.769165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.769931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.770366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.804344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a8dd3e4-b900-42f8-b40f-97d9bb515655 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050213 May 19 12:27:15.805043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 587/1303] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:15.894935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:15.895937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:15.905350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.905826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.906607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:15.907030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:15.968579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:15.976498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-743e42df-f271-4c1c-a203-fad3874dcc9d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088316 May 19 12:27:15.977090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 717/1304] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:16.645774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:16.647279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:16.655306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:16.655844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:16.656592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:16.657049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:16.685593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-23aab7fc-8fa7-4405-9951-b0d4ff4c6806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046545 May 19 12:27:16.686174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 588/1305] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:16.822160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:16.823878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:16.834175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:16.834900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:16.836130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:16.836870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:16.868671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5eee12b-2078-4e25-92f0-8e14352ec1c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053226 May 19 12:27:16.869256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 718/1306] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:16.997644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:16.998661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:17.010245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.011051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.011811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.012240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.075738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:17.083730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0a0d825a-2ecb-4bbc-9985-032ada6a25b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092713 May 19 12:27:17.084326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 589/1307] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:16 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:17.701509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:17.702972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:17.714129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.714509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.715262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.715662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.748121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4560b5fd-7dc7-401f-9bff-8852b3e18e2d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053268 May 19 12:27:17.748693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 719/1308] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:17.884922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:17.886250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:17.888173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=83 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:17.893549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.893994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.894757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:17.895186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:17.924533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-13e52433-218d-468a-b4d0-5b5a519665c6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046633 May 19 12:27:17.925119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 590/1309] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:18.104679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:18.105314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:18.111970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.112409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.113197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.113621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.177180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:18.184888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-89458a61-a6cb-4375-b387-0e9bcb39543a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086653 May 19 12:27:18.185449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 720/1310] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:18.764098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:18.765238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:18.773261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.773815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.774760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.775295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.808673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d147bb50-7c3b-4f15-884e-b1e72eaf206e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050744 May 19 12:27:18.809223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 591/1311] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:18.940469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:18.942000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:18.952934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.953655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.954889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:18.955586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:18.988808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6b6533f5-d286-43f9-9da0-41da213e31e9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054892 May 19 12:27:18.989360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 721/1312] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:19.206818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:19.207821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:19.216619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:19.217097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:19.217883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:19.218282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:19.281363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:19.289118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fb80783f-0bb1-44a0-b31f-d1824b6c039e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088756 May 19 12:27:19.289725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 592/1313] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:19.823661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:19.825214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:19.828100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=86 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:19.836957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:19.837687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:19.838999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:19.839426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:19.869079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5c6c697-3a14-420e-b1e8-c929da4e78cd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052086 May 19 12:27:19.869633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 722/1314] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:20.009478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:20.011581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:20.021680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.022203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.023092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.023604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.057364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-813c0e60-3150-46de-ba34-6dfc4251a57f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059364 May 19 12:27:20.057924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 593/1315] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:20.309499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:20.310465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:20.321591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.322338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.323549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.324037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.395118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:20.405070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e4acdec-7d78-4938-9696-e94040dd6621 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102005 May 19 12:27:20.405643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 723/1316] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:20.885383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:20.886617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:20.893938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.894426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.895256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:20.895727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:20.927493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-14d21406-5439-4b51-82ad-b9507a16dcab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049198 May 19 12:27:20.928039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 594/1317] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:21.073937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:21.074887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:21.081521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.081967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.082719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.083145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.113802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6cdcaf6d-97ac-4068-a842-e378b32915fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046520 May 19 12:27:21.114378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 724/1318] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:21.425216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:21.426219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:21.435489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.435934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.436695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.437146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.504081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:21.512042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fc6e62c7-f71b-4d3c-805e-c2b6c6b2e38b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093344 May 19 12:27:21.512611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 595/1319] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:21.939914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:21.940884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:21.949365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.949974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.950875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:21.951298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:21.981177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35a35449-c5da-4a61-a0ea-f5d76950d5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045551 May 19 12:27:21.981732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 725/1320] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:22.130739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:22.132106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:22.142509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:22.143245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:22.144485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:22.145246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:22.177145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8bbf644b-df94-487c-ab38-440966cdec94 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053856 May 19 12:27:22.177714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 596/1321] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:22.528507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:22.529139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:22.535689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:22.536129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:22.536908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:22.537329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:22.597617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:22.605237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3869d4b8-b6ed-45ee-b8a8-1eab2037c1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.080647 May 19 12:27:22.605764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 726/1322] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:22.997028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:22.998503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:23.009724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.010586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.011363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.011779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.041188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8163aa1f-1d5d-48ec-9392-a6af7da3d86e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050891 May 19 12:27:23.041802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 597/1323] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:23.192517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:23.194010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:23.202724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.203159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.203905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.204325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.235631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5933da71-a68c-47d6-9d0d-c6e9d1b481ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049636 May 19 12:27:23.236178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 727/1324] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:23.623561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:23.624235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:23.631637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.632132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.633002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:23.633484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:23.696240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:23.704565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0705ef6f-331c-4e71-b5ba-b63f0bc2a8ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085855 May 19 12:27:23.705146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 598/1325] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:24.059127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:24.060626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:24.070873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.071598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.073104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.073557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.102491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-72795376-c664-4e10-9bb9-98fa3554135b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050100 May 19 12:27:24.103033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 728/1326] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:24.248218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:24.249154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:24.256437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.256926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.257683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.258130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.293036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e7a6abf9-e3f4-41e5-9fe7-ff3748ed9601 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049192 May 19 12:27:24.293619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 599/1327] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:24.725479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:24.726560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:24.734603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.735049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.735806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:24.736228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:24.798755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:24.806375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77b26078-14ae-4bb4-a838-82213669b127 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087429 May 19 12:27:24.806945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 729/1328] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:25.117907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:25.119385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:25.126027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.126468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.127214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.127634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.159116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7243b583-1460-48b1-8a50-09661da88469 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047898 May 19 12:27:25.159685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 600/1329] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:25.309069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:25.310542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:25.320726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.321536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.322823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.323414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.327097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:25.353285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d88b7c6e-0cb0-4850-8836-d48d44afe35f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050797 May 19 12:27:25.353832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 730/1330] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:25.827752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:25.828831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:25.839551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.839996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.840818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:25.841230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:25.845262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:25.914884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:25.926209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-68f495ad-fc5f-44bb-a793-33f2a8ea9100 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.105212 May 19 12:27:25.927054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 601/1331] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:26.172702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:26.173594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:26.180870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.181338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:26.182124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.182572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:26.213787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1575c307-c0dc-4ba5-b1d8-c6e3beefbcc4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045414 May 19 12:27:26.214343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 731/1332] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:26.434781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:26.435755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:26.442183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.442628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:26.443385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.443809 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:26.473072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07a16da5-cc4f-4f55-bf7b-2282a9b58f44 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.110876 May 19 12:27:26.473617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 602/1333] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:26.949233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:26.949995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:26.957921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.958623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:26.959566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:26.960104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:27.024475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:27.032494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9bac478c-28ce-475c-8316-63647abf8966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089521 May 19 12:27:27.033114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 732/1334] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:27.229726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:27.230966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:27.241099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:27.241782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:27.242923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:27.243578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:27.273329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5958759e-53d9-468d-8a31-3a7c5aecf5f7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050058 May 19 12:27:27.274193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 603/1335] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:27.485774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:27.486682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:27.493627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:27.494072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:27.494828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:27.495253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:27.528397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d1f2423-5cfa-49b0-bb00-9feae1fcfe84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046683 May 19 12:27:27.529012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 733/1336] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:28.053638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:28.054642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:28.062217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.062660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.063423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.063854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.128985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:28.136818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4164e138-ecf8-4138-941f-a80b3491de09 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089609 May 19 12:27:28.137384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 604/1337] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:28.290079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:28.291988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:28.299943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.300385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.301170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.301600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.331952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8a2f3ab9-818a-4c2d-8a0b-a6d767fee182 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048816 May 19 12:27:28.332496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 734/1338] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:28.542361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:28.543280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:28.550362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.550808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.551569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:28.551972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:28.582639 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d9259bf4-0f1d-41c9-9d07-c76caa08c167 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046343 May 19 12:27:28.583117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 605/1339] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:29.157625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:29.158579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:29.168440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.168909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.169653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.170077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.243287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:29.250952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9817c02a-b760-48f2-9c4f-02af7b20b750 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099804 May 19 12:27:29.251485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 735/1340] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:29.346822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:29.348188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:29.356743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.357186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.357935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.358360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.387493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-24abd52a-0062-48b3-9d18-ba724df65a1a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046834 May 19 12:27:29.388045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 606/1341] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:29.595197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:29.596104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:29.602686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.603125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.603867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:29.604290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:29.634491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9fd6243f-f61d-4e27-8258-23fe708cbb0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043319 May 19 12:27:29.635033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 736/1342] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:30.272181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:30.273182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:30.281325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.281770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.282524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.282950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.352128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:30.359849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1a7f143c-91a7-45ae-9d3b-3c2358f5e84c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094082 May 19 12:27:30.360420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 607/1343] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:30.399205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:30.400123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:30.407130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.407569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.408314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.408769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.440005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2d5aead7-27e0-4a9a-8d87-18d0e1239ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044715 May 19 12:27:30.440573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 737/1344] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:30.650230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:30.651744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:30.662095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.662823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.664057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:30.664894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:30.696678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8691c6a9-e21c-426e-b844-ba0a7e12c195 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053036 May 19 12:27:30.697269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 608/1345] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:31.383600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:31.384626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:31.391443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.391883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.392659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.393094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.454458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:31.455795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:31.456278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:31.463809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-220e2dbd-ba72-469d-956a-d6534e1014ae tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086975 May 19 12:27:31.464087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.473386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.474701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 738/1346] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:31.474951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.474951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.496288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-faf3f4c1-38ed-4c6d-be4c-5b15ec89a3be tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048419 May 19 12:27:31.496878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 609/1347] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:31.711893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:31.713244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:31.719887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.720333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.721127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:31.721555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:31.751253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd7ce37a-573a-4314-b470-1a87e0c0ae1d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046153 May 19 12:27:31.751813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 739/1348] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:32.485435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:32.486483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:32.496669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.497159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.497923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.498350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.510506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:32.511422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:32.517900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.518358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.519093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.519646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.549726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86f2c26a-d34f-41ff-9c84-572e9de258cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045976 May 19 12:27:32.550317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 740/1349] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:32.565019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:32.572715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3546d5cb-77e5-4659-9fbf-1ac3979b193d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093847 May 19 12:27:32.573303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 610/1350] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:32.765275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:32.766102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:32.772626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.773109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.773865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:32.774291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:32.803765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e7fbda1-eee1-4928-8fcd-096a4b9de5bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042927 May 19 12:27:32.804309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 741/1351] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:33.565693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:33.567176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:33.576990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.577430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.578184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.578606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.589481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:33.590086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:33.598401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.599150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.600094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.600505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.609871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab6f7034-076f-4507-a91e-97b43c2c3e4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050744 May 19 12:27:33.610441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 611/1352] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:33.665446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:33.673554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be480457-32a5-463c-aa69-0d3c7418a091 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087888 May 19 12:27:33.674118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 742/1353] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:33.820884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:33.822366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:33.832879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.833645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.834899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:33.835611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:33.875017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e63c5322-ef72-4b34-a4a9-7822763e40c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060761 May 19 12:27:33.875584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 612/1354] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:34.624511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:34.625514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:34.633150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.633606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.634388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.634833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.668355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-78049e5a-326f-4c34-8560-7a774c7637fd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048880 May 19 12:27:34.668972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 743/1355] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:34.691396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:34.692003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:34.698957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.699409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.700167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.700590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.762945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:34.770661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cb3d9bfe-58c5-4252-9c75-52a779e540ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083132 May 19 12:27:34.771225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 613/1356] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:34.891614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:34.893149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:34.904021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.904458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.905236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:34.905679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:34.936770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-848acd5e-021d-4138-887c-10bd6814c912 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052186 May 19 12:27:34.937353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 744/1357] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:35.679876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:35.680727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:35.687232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.687683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.688421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.688878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.717427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d284ea6e-d6e3-4c31-814b-650c0dd739af tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041658 May 19 12:27:35.718062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 614/1358] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:35.791726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:35.792841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:35.800410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.800883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.801639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.802101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.874775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:35.884678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e17c126c-f35c-45bf-a995-3e00b04b2259 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098877 May 19 12:27:35.884678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 745/1359] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:35.950684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:35.951529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:35.957942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.958452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.959217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:35.959651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:35.988937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e891cd46-606b-490c-9416-67aa6a57b48b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042454 May 19 12:27:35.989508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 615/1360] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:36.729611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:36.730468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:36.737277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:36.737711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:36.738462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:36.738889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:36.765483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:36.770152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d1b34f2-ad42-4030-9583-c64614ba6650 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044736 May 19 12:27:36.770706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 746/1361] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:36.905787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:36.906528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:36.915886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:36.916480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:36.917537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:36.918114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:36.991465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:36.999388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c12f3c41-6996-4a23-aeec-9e11ea4fb300 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099929 May 19 12:27:37.000163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 616/1362] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:37.000804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:37.001820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:37.009628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:37.010158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:37.011290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:37.011929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:37.047789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ca185c91-a770-45dc-b0c7-2b978aa20239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051046 May 19 12:27:37.048352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 747/1363] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:37.783354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:37.784284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:37.791463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:37.791941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:37.792825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:37.793293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:37.819472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:37.824108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-658d82d1-ddbd-4f5c-a459-e3d8649b1f0c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045349 May 19 12:27:37.824647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 617/1364] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:38.021493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:38.022516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:38.033068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.033950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.035070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.035821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.059667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:38.062110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:38.067834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.068281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.069065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.069489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.099289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f4657839-7791-4f94-9027-9f38c9054eb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043522 May 19 12:27:38.099856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 618/1365] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:38.105300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:38.113037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7da28031-3050-4cc8-8d37-d110e3ca5283 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097989 May 19 12:27:38.113607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 748/1366] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:38.914793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:38.915716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:38.922289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.922733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.923483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:38.923916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:38.954110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aee8ba7a-64f2-4bb5-bb8b-7eca03aa9c54 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.120973 May 19 12:27:38.954674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 619/1367] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:39.115486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:39.116902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:39.126120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.126560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:39.127313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.127736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:39.130254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:39.130887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:39.137713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.138160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:39.138941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.139367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:39.157519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5f70fd8-a692-4ccf-a56d-894a7c76b915 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048764 May 19 12:27:39.161645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 749/1368] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:39.206967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:39.215004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de0e610c-61bd-405b-b73b-026ea2563652 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088863 May 19 12:27:39.215587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 620/1369] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:39.969347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:39.970412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:39.978055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.978576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:39.979451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:39.979947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:40.012250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a32901d-1189-425a-8f02-0cdef496794a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048713 May 19 12:27:40.012824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 750/1370] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:40.172468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:40.173416 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:40.180621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:40.181098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:40.181855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:40.182289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:40.213132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cd2148ad-3560-4859-96c0-3706316c9663 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045901 May 19 12:27:40.213683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 621/1371] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:40.233642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:40.234155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:40.241187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:40.241646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:40.242366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:40.242788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:40.358998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:40.366703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb19f588-d1a1-4304-a673-f9b71255b448 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.138957 May 19 12:27:40.367275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 751/1372] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:41.025104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:41.026049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:41.032426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.032980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.033751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.034168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.063684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-26ababae-e8a5-4bb8-8192-94b96d4ddc83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043942 May 19 12:27:41.064259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 622/1373] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:41.228721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:41.230040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:41.237960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.238554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.239140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.239567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.269308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8f63e35b-746b-4bb1-8c1a-9557476e7ac5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047536 May 19 12:27:41.269864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 752/1374] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:41.387818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:41.388874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:41.399741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.400485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.401775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:41.402481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:41.530235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:41.538299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b8e04ae-a804-4234-8eb7-b4422716a5b5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.156912 May 19 12:27:41.538894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 623/1375] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:42.075743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:42.077371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:42.083083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.083521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.084267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.084707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.116726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db667b28-c518-49e7-962e-0ff6aa8f078b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044924 May 19 12:27:42.117315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 753/1376] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:42.283419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:42.284339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:42.291079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.291625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.292441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.292920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.322699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5518c167-b369-4fd5-b209-69ee3780f497 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043352 May 19 12:27:42.323244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 624/1377] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:42.555666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:42.556230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:42.563271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.563709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.564451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:42.564909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:42.625486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:42.633359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f200993-c0f1-4dd1-8d30-ef2d6a1e7f7e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.081633 May 19 12:27:42.633932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 754/1378] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:43.133701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:43.135216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:43.148720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.149499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.150810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.151516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.182007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-601982fb-201f-4bc0-b155-c4ca5efb7fd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055541 May 19 12:27:43.182551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 625/1379] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:43.339198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:43.340623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:43.351309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.352011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.353549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.355225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.386338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b3289dd4-d39e-453c-9865-2907b03d2a2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053813 May 19 12:27:43.386810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 755/1380] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:43.654371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:43.655004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:43.662453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.662972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.663854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:43.664326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:43.735962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:43.744039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a4683160-f39d-421d-b065-bdfd5ae1d099 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096404 May 19 12:27:43.744605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 626/1381] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:44.193871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:44.195428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:44.202289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.203134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.203501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.203919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.234495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-de7acaf8-a3f4-4a92-8bca-5791f7f8cedc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044566 May 19 12:27:44.235300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 756/1382] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:44.401177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:44.402370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:44.410394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.410896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.411596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.412028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.441189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-35c44f03-e950-4f49-bf1c-724b058edbe5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046364 May 19 12:27:44.441743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 627/1383] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:44.764839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:44.765401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:44.772226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.772697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.773547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:44.773993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:44.852018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:44.860018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3c4b5a78-1b0c-4131-a6d6-36112a1e7ae9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101602 May 19 12:27:44.860592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 757/1384] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:45.246331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:45.247303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:45.254266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.254814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.255681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.256142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.286296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-33c6ccb5-7bf5-4b09-9f74-4d3337294655 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043843 May 19 12:27:45.286857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 628/1385] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:45.459065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:45.460414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:45.471219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.471659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.472443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.472907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.508002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b6f08d86-0791-429b-8b01-20ee2a6b06b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055806 May 19 12:27:45.508604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 758/1386] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:45.881409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:45.882436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:45.893511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.893951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.894708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:45.895125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:45.963916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:45.971617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7657e99a-e98b-4959-9793-f00685d37636 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096954 May 19 12:27:45.972182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 629/1387] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:46.301996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:46.303365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:46.313750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:46.314485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:46.315754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:46.316204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:46.345719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-acc4a017-b10b-43c7-8eaf-e2e2f8de0e98 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050495 May 19 12:27:46.346357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 759/1388] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:46.521831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:46.522750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:46.529251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:46.529689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:46.530470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:46.530893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:46.561324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-591c3e13-12d0-4151-8565-d1dae08e3b91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044210 May 19 12:27:46.561875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 630/1389] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:46.992775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:46.993651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:47.004912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.005353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.006108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.006533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.075152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:47.082984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f3d77d6e-8893-4fff-a65d-4dc53bcb9f9a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096905 May 19 12:27:47.083557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 760/1390] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:46 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:47.362032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:47.363517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:47.370680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.371120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.371867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.372290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.401725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e8e8077a-640c-4e07-860b-a41b7ed7d1c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046551 May 19 12:27:47.402278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 631/1391] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:47.577440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:47.578923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:47.589486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.590257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.591497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:47.592202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:47.624349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-13e6857d-a11d-4263-b8f2-7a29cd7b778e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053560 May 19 12:27:47.624869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 761/1392] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:48.103760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:48.104926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:48.107928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=84 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:48.115785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.116222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.117027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.117426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.178185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:48.185830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bfa6985c-f650-4b7f-9d01-34a4d6233cfa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088461 May 19 12:27:48.186381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 632/1393] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:48.415149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:48.416302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:48.425893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.426524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.427592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.428198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.461197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e0efe4ab-50ed-478c-97ad-20ae633309d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051512 May 19 12:27:48.461744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 762/1394] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:48.639606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:48.641112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:48.651238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.651979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.653259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:48.653964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:48.685638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-66865d9b-aa69-4569-a626-f27fca5fe0dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052616 May 19 12:27:48.686113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 633/1395] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:49.206596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:49.207640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:49.217672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.218416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.219650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.220363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.241049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:49.294673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:49.306613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a4dd4e4-b6dc-4b4b-92c5-f31f58bf362b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.106180 May 19 12:27:49.307406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 763/1396] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:49.477507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:49.478912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:49.489594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.490046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.490845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.491275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.508624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:49.523996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7d4ee232-8883-4444-883f-68606c7ef961 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053317 May 19 12:27:49.524596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 634/1397] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:49.701519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:49.702959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:49.711581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.712021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.712805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:49.713228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:49.743147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-428ef054-28c7-4478-9b16-14700ed6af2e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048334 May 19 12:27:49.743620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 764/1398] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:50.327653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:50.328876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:50.336758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.337193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.337937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.338385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.402162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:50.410311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dc146b8c-5662-47da-be00-0f5ca5716865 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088737 May 19 12:27:50.411306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 635/1399] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:50.538031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:50.538957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:50.541677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=86 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:50.548093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.548545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.549330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.549764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.580883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7e3bc522-5c57-425d-8f4a-b6774f99ab10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048759 May 19 12:27:50.581375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 765/1400] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:50.758342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:50.759415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:50.767425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.767946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.768868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:50.769373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:50.801904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-067dfe8b-7ad4-4f4a-868a-94630dff0317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048709 May 19 12:27:50.802479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 636/1401] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:51.434055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:51.434977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:51.444455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.444925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.445673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.446095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.508967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:51.516690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fd635fa1-4edf-40d0-b54b-e2649df165b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089171 May 19 12:27:51.517260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 766/1402] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:51.594997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:51.596101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:51.603112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.603556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.604302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.604761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.634355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f24b33b4-7863-4542-80ff-5981cf54814e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045326 May 19 12:27:51.634904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 637/1403] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:51.814527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:51.815458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:51.822202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.822638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.823418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:51.823836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:51.857638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3c416b56-5e05-49f4-be86-a266a2d861c4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047215 May 19 12:27:51.858210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 767/1404] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:52.534486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:52.535093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:52.542275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.542718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.543471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.543897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.617952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:52.625770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b8303f1f-a4d4-426f-b385-4698608706ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095197 May 19 12:27:52.626339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 638/1405] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:52.647304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:52.648232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:52.655213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.655656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.656422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.656935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.689575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2806e9c3-aa71-47e3-bf90-fe5ec71c2a5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046427 May 19 12:27:52.690147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 768/1406] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:52.872596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:52.873695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:52.880733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.881169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.881941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:52.882361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:52.918404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-677e950a-84d8-47c5-87a2-4f4e9f957d21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051969 May 19 12:27:52.919033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 639/1407] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:53.643905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:53.644775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:53.655246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.655859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.656605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.657061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.701701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:53.702693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:53.709933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.710372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.711124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.711560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.724618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:53.732259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e03b8201-dfe0-4e05-9a5d-5f183e55d689 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092605 May 19 12:27:53.732825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 769/1408] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:53.741594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0217e4b0-ea16-4904-ab10-048afef34207 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043830 May 19 12:27:53.742029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 640/1409] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:53.935763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:53.937304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:53.945494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.945933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.946674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:53.947096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:53.979679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e39d04d3-7bf0-4fd3-b553-8e9a2beb0d36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049418 May 19 12:27:53.980249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 770/1410] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:54.753665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:54.754744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:54.756517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:54.758034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:54.765798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:54.766431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:54.767671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:54.768375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:54.768675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:54.769168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:54.770230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:54.770659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:54.801635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c68c62fa-9b6e-4f0c-9b8f-163b07b80b49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051871 May 19 12:27:54.802207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 771/1411] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:54.833849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:54.841807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-24a8b51c-778e-4379-a12f-bab5d1830eea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095146 May 19 12:27:54.842407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 641/1412] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:54.996105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:54.997641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:55.008324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.009150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.010415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.011127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.044795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35f89b61-1ea7-453a-8e09-b92c7fe0003c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055400 May 19 12:27:55.045353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 772/1413] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:55.817279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:55.818197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:55.825092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.825546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.826293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.826718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.860202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:55.860825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-123fcd75-0512-489a-b190-672a9c8ad530 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049258 May 19 12:27:55.860825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 642/1414] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:55.861054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:55.872412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.873041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.874177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:55.874792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:55.948548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:55.956169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63c76d3c-c7d4-4f18-86b7-fef940a8563f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101288 May 19 12:27:55.956752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 773/1415] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:55 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:56.056147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:56.057121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:56.063559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.064003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:56.064787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.065214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:56.095232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2a02cd00-08e1-4bbf-85cc-14f70aa71ee4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043065 May 19 12:27:56.095742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 643/1416] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:56.875645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:56.877699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:56.885178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.885582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:56.886368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.886788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:56.916464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e058b07-6f20-4f06-9816-ffeb558357f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047768 May 19 12:27:56.917061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 774/1417] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:56.977281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:56.978248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:56.989149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.989585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:56.990345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:56.990773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:57.061267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:57.073148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d2673270-e6ef-46ee-9895-4b7310f91997 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102062 May 19 12:27:57.073893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 644/1418] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:57.107083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:57.108003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:57.115579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:57.116118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:57.117020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:57.117426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:57.147425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aea03b23-19a1-46a9-b7f2-6d119ed1a1ad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044247 May 19 12:27:57.147905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 775/1419] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:57.931020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:57.931986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:57.938492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:57.938946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:57.939714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:57.940144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:57.969846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-514058ac-993a-42d0-929e-42c482d689f5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043938 May 19 12:27:57.970467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 645/1420] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:58.093768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:58.094399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:58.103132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.103573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:58.104318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.104769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:58.159751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:58.161008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:58.168185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.168659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:58.169450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.169887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:58.175831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:58.184038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18cd787a-5823-44dd-aa84-fdca3dc8b878 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096729 May 19 12:27:58.184566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 776/1421] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:27:58.200519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b0ab4de6-eb50-46bd-bfbf-f032275b9a2a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045513 May 19 12:27:58.201129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 646/1422] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:27:58.982004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:58.982983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:58.989674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.990119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:58.990880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:58.991306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:59.024218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f88b3674-6231-4a3c-bc58-38edc5566e35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046240 May 19 12:27:59.024891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 777/1423] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:27:59.201660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:59.202293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:59.209783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:59.210284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:59.211747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:59.211895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:27:59.212293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:59.212405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:27:59.219202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:59.219641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:59.220408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:27:59.220884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:27:59.251403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e3ecc1e-68ea-4499-a37c-3fba099516a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043830 May 19 12:27:59.251849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 778/1424] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:27:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:27:59.275317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:27:59.282946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a3863b1-f486-417c-991e-32f053a718a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085606 May 19 12:27:59.283516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 647/1425] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:27:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:00.040995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:00.042551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:00.050463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.050905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.051740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.052215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.084200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3c994847-7eb1-498b-b806-b072ddb0b34d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049821 May 19 12:28:00.084782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 779/1426] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:00.268595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:00.270484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:00.281641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.282259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.283368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.284016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.299587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:00.300118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:00.307234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.307731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.308511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:00.308968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:00.313162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:00.322900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3afb7f4a-2942-4e9f-9e48-7c23cfb5c5b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062560 May 19 12:28:00.323498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 648/1427] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:00.380471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:00.393240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-95c0a135-1845-49d0-9a81-f3a0b6452e68 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097379 May 19 12:28:00.394162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 780/1428] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:01.100483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:01.102075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:01.112358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.113146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.114384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.115092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.146987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afd7206d-6252-4ff9-bfa2-38e3f7018560 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053093 May 19 12:28:01.147533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 649/1429] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:01.335026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:01.335968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:01.343426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.343868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.344620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.345085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.375053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-795196d8-3582-430d-94cf-e4df9b97e76f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044264 May 19 12:28:01.375669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 781/1430] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:01.413397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:01.414078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:01.421271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.421717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.422489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:01.422919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:01.495427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:01.503985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d556ea36-7ad5-4dbc-8e8b-5d0f836f6c19 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094395 May 19 12:28:01.504601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 650/1431] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:02.165023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:02.166897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:02.175976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.176465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.177378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.177883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.209781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c8304cf2-2d67-4076-8aeb-43828e920fe1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053348 May 19 12:28:02.210335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 782/1432] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:02.387946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:02.388893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:02.395317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.395762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.396509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.396973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.400567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:02.427700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38dbdf0c-7c27-4941-94f1-a93ce0476259 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043636 May 19 12:28:02.428336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 651/1433] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:02.521279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:02.521888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:02.528790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.529231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.530017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:02.530442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:02.595324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:02.602900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a297dc7-6ac6-4438-a418-4f5ab96384d0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085670 May 19 12:28:02.603466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 783/1434] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:03.223742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:03.224575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:03.230922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.231371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.232126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.232555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.266628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-370d2592-31c3-43f0-9249-4884c68dff20 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048163 May 19 12:28:03.267266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 652/1435] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:03.443101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:03.444024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:03.450488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.450937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.451698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.452127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.481864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-07b0a410-a727-42cd-9ed9-1e88968f550e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042976 May 19 12:28:03.482353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 784/1436] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:03.620545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:03.621180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:03.628663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.629108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.629867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:03.630293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:03.702336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:03.711619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5933208c-83b1-40eb-8df2-df1917803e0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095215 May 19 12:28:03.712180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 653/1437] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:04.280629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:04.281792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:04.290276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.290827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.291775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.292301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.324127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63bdcaad-3bf7-4c47-82e1-528340bfcbe7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048501 May 19 12:28:04.324631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 785/1438] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:04.494137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:04.495039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:04.501683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.502506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.503264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.503685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.539697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-963ff95e-ca71-4282-a7c2-cc056f35cc1a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049492 May 19 12:28:04.540272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 654/1439] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:04.729034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:04.729647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:04.736504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.736981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.737738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:04.738159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:04.804446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:04.812923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9b9afd7d-b8ec-49da-b2b3-35e1da8c2ba3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088042 May 19 12:28:04.813524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 786/1440] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:05.337484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:05.338480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:05.348421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.349103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.349870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.350300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.381254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-732feb0f-a7ac-4531-a489-c3a7d8dca545 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049082 May 19 12:28:05.381814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 655/1441] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:05.554065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:05.554982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:05.561811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.562257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.563011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.563436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.593671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4953b4f5-b3fe-4ca5-b805-6d1e78ec385f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045369 May 19 12:28:05.594140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 787/1442] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:05.831435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:05.832171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:05.839223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.839666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:05.840874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:05.912982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:05.922097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f93d4444-550b-4e0f-9c6c-8a3ece87db18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095215 May 19 12:28:05.922671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 656/1443] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:06.393536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:06.394485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:06.401451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.401907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:06.402674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.403101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:06.432984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8940c8d1-efab-4d19-99d1-e26d52c8ce47 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043620 May 19 12:28:06.433538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 788/1444] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:06.612012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:06.613006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:06.621475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.622008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:06.622832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.623278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:06.658640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-213ea94b-c14a-46ed-afd8-a348456cb1f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052291 May 19 12:28:06.659117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 657/1445] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:06.942859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:06.943491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:06.950340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.950798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:06.951543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:06.951978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:07.014657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:07.022324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2cd44503-82e1-433b-bfcc-c0dacc154093 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083477 May 19 12:28:07.022898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 789/1446] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:07.449207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:07.450173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:07.456936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:07.457393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:07.458175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:07.458608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:07.490054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-500d1efd-20f0-4ee7-bc56-c645fddfc275 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046268 May 19 12:28:07.490657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 658/1447] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:07.675646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:07.677172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:07.685966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:07.686403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:07.687233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:07.687699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:07.724071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-608bfe64-520d-43e4-8680-4ea5de8deb91 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055407 May 19 12:28:07.724534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 790/1448] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:08.042538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:08.043365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:08.050975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.051415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.052171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.052595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.120290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:08.132714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b5c98835-ed93-41f3-914a-c54e9118704e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096105 May 19 12:28:08.133664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 659/1449] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:08.503388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:08.504585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:08.511382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.511824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.512591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.513046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.543848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3c2e0706-1d89-4f6c-a4a2-5a757268782c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045190 May 19 12:28:08.544441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 791/1450] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:08.740582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:08.741435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:08.748034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.748480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.749270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:08.749697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:08.780975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e50f10e0-9d23-4559-bcd4-6a5cc4031ec1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044648 May 19 12:28:08.781565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 660/1451] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:09.157137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:09.157755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:09.164885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.165328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.166087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.166519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.237269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:09.248933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e49bdd6f-e881-4fa0-8010-8a4ffc635521 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097982 May 19 12:28:09.249711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 792/1452] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:09.557568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:09.558482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:09.565666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.566109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.566871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.567288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.597100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-533c4896-5e34-4dad-ae85-d93bcd693c51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043659 May 19 12:28:09.597695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 661/1453] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:09.796905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:09.797907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:09.807340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.807916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.809093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:09.809676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:09.840506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b7afcffb-194c-493d-a3e1-ac52a4bd698a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050106 May 19 12:28:09.841199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 793/1454] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:10.270031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:10.270659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:10.277391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.277840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.278607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.279041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.343751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:10.352032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-caa90be4-6697-4447-b024-1bdc31b81b3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088009 May 19 12:28:10.352612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 662/1455] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:10.614374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:10.615984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:10.627902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.628743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.629789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.630207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.666349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fa071093-e89b-41d8-837d-99b433613b85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059209 May 19 12:28:10.667012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 794/1456] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:10.855455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:10.856700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:10.865875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.866263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.867041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:10.867472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:10.898185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba5c5441-c3c7-4ed6-b03e-03ac4d36cd1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048271 May 19 12:28:10.899027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 663/1457] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:11.378197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:11.379150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:11.386006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.386456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.387212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.387641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.459769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:11.467599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b11c4bf3-6d34-4707-97df-6f2fa9581ad5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096338 May 19 12:28:11.468163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 795/1458] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:11.682930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:11.684303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:11.695311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.696048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.697407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.698123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.731950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1d7855a8-f1ac-4043-ba7c-10b5aae23a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055881 May 19 12:28:11.732497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 664/1459] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:11.914607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:11.916071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:11.926415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.926878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.927626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:11.928047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:11.953831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:11.958448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-47353e4c-48d9-449e-8b2c-f680c5e78828 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050579 May 19 12:28:11.959012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 796/1460] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:12.489078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:12.489669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:12.496624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.497103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:12.497865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.498302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:12.563568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:12.571541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1c77ee55-88fd-4cb9-ae5b-0fb465e5ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089384 May 19 12:28:12.572121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 665/1461] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:12.748211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:12.749610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:12.759511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.759954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:12.760744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.761174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:12.791053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11850441-8ce1-450b-8c51-454882bf7aa0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049725 May 19 12:28:12.791774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 797/1462] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:12.972541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:12.973853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:12.980405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.980897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:12.981661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:12.982097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:13.012454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-69a72abc-664e-4f3d-8f5a-5f593e11e5cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045387 May 19 12:28:13.012974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 666/1463] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:13.590023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:13.590693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:13.598479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:13.598961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:13.599755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:13.600219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:13.803524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:13.804474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:13.811763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:13.812212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:13.813024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:13.813457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:13.840166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:13.845215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a5274cc6-0b06-4226-bc3d-7d4342991019 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046040 May 19 12:28:13.845807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 667/1464] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:13.898952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:13.909099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8efe3381-4cb5-4e63-a65c-893b591602fa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.323343 May 19 12:28:13.909675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 798/1465] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:14.026392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:14.028820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:14.037111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.037733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:14.038786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.039182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:14.069010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e2d18939-c40d-4ffe-80f6-045064b1e982 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048338 May 19 12:28:14.069714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 668/1466] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:14.861379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:14.862924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:14.872450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.872922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:14.873645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.874088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:14.908235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7c1eaeca-2bc4-44cd-946b-d85a85589eb1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053589 May 19 12:28:14.908912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 799/1467] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:14.926133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:14.926742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:14.933670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.934112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:14.934859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:14.935285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:15.006534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:15.014152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d27ff23a-78a1-4daf-b57c-c37b273edb41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091961 May 19 12:28:15.014749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 669/1468] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:15.084923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:15.086410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:15.093800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:15.094243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:15.094996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:15.095427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:15.124517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ade63a8-22bc-456c-91a1-b9e16d041bc2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046141 May 19 12:28:15.125067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 800/1469] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:15.922224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:15.923150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:15.929503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:15.930109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:15.930680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:15.931107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:15.961739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-23d7174e-72e8-4bc8-94e3-8b3623951029 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044189 May 19 12:28:15.962336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 670/1470] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:16.032732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:16.033285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:16.040497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.041298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:16.043807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.044290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:16.118035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:16.130068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5026b40-b64a-41dc-bb81-69ab4fc2434b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102110 May 19 12:28:16.130933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 801/1471] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:16 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:16.138448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:16.139368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:16.145713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.146159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:16.146900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.147320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:16.175919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-914e43c3-c44b-4485-8e1d-39bc4e71c870 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041697 May 19 12:28:16.176435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 671/1472] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:16.978519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:16.979983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:16.987607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.988021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:16.988813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:16.989245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:17.018726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f9a99f18-e456-4e89-8f5b-425e962b326f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047086 May 19 12:28:17.019244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 802/1473] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:17.147099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:17.147749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:17.155150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:17.155649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:17.156439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:17.156904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:17.187611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:17.188521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:17.195504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:17.195947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:17.196726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:17.197179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:17.220294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:17.227401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c617bdce-fa11-48f3-8144-f37d0a38abb6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043726 May 19 12:28:17.227957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 803/1474] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:17.228660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62a1f43c-de29-4745-82cf-f780b851e225 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085674 May 19 12:28:17.229226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 672/1475] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:18.033427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:18.034444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:18.041499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.041996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.042826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.043303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.081704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92aad529-a8d3-4bc1-a5c8-db99f9a2d7d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053757 May 19 12:28:18.082491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 804/1476] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:18.240460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:18.241412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:18.243477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=82 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:18.245353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:18.245970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:18.249285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.249713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.250522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.250947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.253082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.253518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.254264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:18.254685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:18.282018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d3a6eff2-0551-450d-ba48-6d9bc9a4d688 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045723 May 19 12:28:18.282587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 673/1477] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:18.318016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:18.325779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb2af569-9429-409a-a661-93932e623bd6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084331 May 19 12:28:18.326376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 805/1478] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:19.096664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:19.097711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:19.105305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.105827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.106710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.107208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.137610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19360db9-8506-4b82-8292-c47a38389e8a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045726 May 19 12:28:19.138164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 674/1479] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:19.296560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:19.297891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:19.307984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.308572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.309810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.310477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.342084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9cdd54c2-9630-458e-984d-ac627385b520 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051865 May 19 12:28:19.342659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 806/1480] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:19.343257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:19.343866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:19.351807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.352176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.353117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:19.353541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:19.418960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:19.427775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a6863046-c6df-476b-a11b-ce51736cbb07 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088830 May 19 12:28:19.428355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 675/1481] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:20.152392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:20.153334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:20.159830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.160271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.161054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.161486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.190242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-232f4f9a-a321-412a-9ed8-cb31cca89895 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041934 May 19 12:28:20.190786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 807/1482] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:20.355084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:20.356234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:20.364314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.364850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.365748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.366226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.398065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-283f9855-c4fd-49fb-a30b-28511316c8b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047679 May 19 12:28:20.398636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 676/1483] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:20.450412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:20.450974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:20.459733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.460270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.461504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:20.462060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:20.541961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:20.553775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3bd860f1-45bd-485b-a686-1f717962b1ce tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.107601 May 19 12:28:20.554525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 808/1484] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:21.202546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:21.203465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:21.209748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.210204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.210960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.211393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.240483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3f9ef1e6-f275-40db-a50b-e77e5be0be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042095 May 19 12:28:21.241085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 677/1485] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:21.411067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:21.412020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:21.414032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=88 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:21.419839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.420278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.421142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.421590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.451700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c036d2fa-f284-4716-8125-960c773f2609 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044768 May 19 12:28:21.452451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 809/1486] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:21.571636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:21.572579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:21.581558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.581999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.582771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:21.583194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:21.662983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:21.673076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9154db43-0f9d-4db7-813b-d18d67a624c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.105355 May 19 12:28:21.674381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 678/1487] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:22.256026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:22.256979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:22.263996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.264554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.265337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.265769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.297086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2418045e-0044-4c1b-b721-7476685080c5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047421 May 19 12:28:22.297627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 810/1488] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:22.464551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:22.465867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:22.475750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.476295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.477155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.477710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.509574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b158391c-a92a-4c3e-ac43-a7bf01b2a5c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049003 May 19 12:28:22.510169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 679/1489] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:22.691916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:22.693237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:22.699734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.700179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.701037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:22.701463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:22.764979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:22.772851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-afcca592-50ff-473d-a2f6-1dcbab6d825c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084811 May 19 12:28:22.773419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 811/1490] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:23.311483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:23.312678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:23.320870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.321371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.322254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.322739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.356216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a8ed6916-cb04-4fb0-91c8-ef303ea6512f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049217 May 19 12:28:23.357012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 680/1491] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:23.523520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:23.524441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:23.531424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.531869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.532669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.533193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.563039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-af29f267-61ae-46c7-811e-08f1f6e07317 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043512 May 19 12:28:23.563564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 812/1492] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:23.790472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:23.791127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:23.797995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.798402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.799152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:23.799577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:23.870077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:23.878799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-475c8101-ddcd-4880-91ff-384d17e4aea9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092456 May 19 12:28:23.879375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 681/1493] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:24.371281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:24.372346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:24.380230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.380671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.381466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.381891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.398704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:24.412432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b3d9973b-4fc8-454b-b7af-1aaa0cb5a6e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045964 May 19 12:28:24.413030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 813/1494] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:24.575951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:24.576887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:24.584140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.584580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.585373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.585823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.615676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e736116-2640-4a74-ac35-709a2d8225b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043901 May 19 12:28:24.616251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 682/1495] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:24.902124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:24.902730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:24.909807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.910242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.911090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:24.911511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:24.974060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:24.982093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3465eff6-d0e8-405e-8338-b4c15f6fd919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083858 May 19 12:28:24.982669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 814/1496] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:25.424683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:25.425624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:25.432103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:25.432540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:25.433313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:25.433738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:25.463453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c868c4b6-867f-4c8d-89ba-5bdf6531aade tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042812 May 19 12:28:25.463984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 683/1497] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:25.631093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:25.632019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:25.638384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:25.638822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:25.639567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:25.639993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:25.670543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd97ad21-90e6-45e9-a945-24e76f8b67f2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044617 May 19 12:28:25.671107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 815/1498] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:26.003865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:26.004920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:26.016753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.017575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.018911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.019712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.038275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:26.083465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:26.092103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-632b547a-a99c-445f-804d-7f7dfee0178e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095442 May 19 12:28:26.092683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 684/1499] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:26.479291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:26.480494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:26.490911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.491526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.492670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.493209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.525234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-70b3bd37-39fd-403a-bfbf-d24229ba8b09 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052896 May 19 12:28:26.525831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 816/1500] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:26.685985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:26.687326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:26.696771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.697222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.697991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:26.698424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:26.728630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a9fe94b3-ee38-4e5c-a414-17a06473b3fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049365 May 19 12:28:26.729230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 685/1501] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:27.115148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:27.115999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:27.123396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.123859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.124657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.125078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.195614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:27.203780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-201b2743-af53-4f52-b73e-f8a364086b17 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093259 May 19 12:28:27.204390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 817/1502] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:27.537552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:27.538499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:27.545515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.545954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.546707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.547127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.577054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-971816c1-2d1c-4182-8c3a-07bfcba7bc77 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043524 May 19 12:28:27.577625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 686/1503] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:27.748687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:27.749669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:27.757301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.757678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.758421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:27.758842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:27.797177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-982e31b6-b632-4ef9-a9cc-2a66ae422cda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058638 May 19 12:28:27.797875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 818/1504] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:28:28.223080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:28.223790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:28.231464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.231985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.232970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.233398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.305682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:28.317581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-831852e6-16bd-46a7-b68a-48128bf476f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098357 May 19 12:28:28.318198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 687/1505] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:28.593391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:28.594287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:28.600958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.601403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.602158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.602587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.633189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65a53aae-26dc-423e-970a-8904dd159117 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045600 May 19 12:28:28.633741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 819/1506] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:28.816168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:28.817837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:28.828299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.828775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.829696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:28.830154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:28.860861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0d87fbe4-0bc9-4590-b688-368f90411d02 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051753 May 19 12:28:28.861415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 688/1507] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:29.339659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:29.340386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:29.347498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.347941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.348679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.349131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.415848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:29.424074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e0ebad4f-6feb-4a09-9409-f6c8d965bfcb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088314 May 19 12:28:29.424658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 820/1508] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:29.649529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:29.650714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:29.659810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.660457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.661586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.662213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.694999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6b942b60-2baa-4e8e-b919-1fb9dcbfd547 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051411 May 19 12:28:29.695564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 689/1509] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:29.877337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:29.878264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:29.884862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.885312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.886069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:29.886500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:29.917868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9da9c81a-b9ea-44aa-971f-d460c210a4ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047326 May 19 12:28:29.918354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 821/1510] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:30.441606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:30.442256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:30.449039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.449492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.450252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.450681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.524546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:30.532310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62f1597d-b024-4409-860e-75a649516685 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094831 May 19 12:28:30.532887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 690/1511] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:30.707861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:30.708674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:30.717782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.718373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.719282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.719805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.752152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3deb10a6-9cae-43a1-a9f2-12d230eb1fa6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048473 May 19 12:28:30.752742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 822/1512] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:30.932298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:30.933325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:30.939918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.940369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.941170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:30.941598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:30.971626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2d19582a-c87f-41e5-a1f2-52dd8af6e541 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043736 May 19 12:28:30.972113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 691/1513] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:31.548680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:31.549302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:31.555923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:31.556370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:31.557149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:31.557705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:31.631066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:31.639138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6977f037-6d5a-49a4-bfa6-1d6dd8419b0d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094409 May 19 12:28:31.639720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 823/1514] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:31.768351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:31.769743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:31.779101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:31.779547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:31.780337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:31.780792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:31.810117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-465140b5-7834-49b3-a0e1-2eb9f5dbda82 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048552 May 19 12:28:31.810670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 692/1515] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:31.988593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:31.990148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:32.001478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.002283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.003521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.004225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.034665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eb1fc901-88c2-4b80-9e43-6f3bb997612e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052677 May 19 12:28:32.035140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 824/1516] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:32.659754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:32.660726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:32.667679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.668118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.668935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.669359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.737917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:32.745967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0973db3b-762f-47f2-bc6d-3ea9c33a2b92 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092535 May 19 12:28:32.746527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 693/1517] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:32.823793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:32.824787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:32.831248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.831686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.832697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:32.833147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:32.863880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-97cad210-ee83-43b8-b51b-587158b88ba2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044444 May 19 12:28:32.864412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 825/1518] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:33.050806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:33.052315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:33.059443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.059886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.060656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.061090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.091655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bb87b9e6-09a3-42e6-aafa-d9fb969ecc92 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047441 May 19 12:28:33.092154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 694/1519] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:33.766917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:33.767774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:33.776356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.776830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.777571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.777987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.843531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:33.851301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dd10d301-52b2-4b86-bc18-c108a109b839 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091021 May 19 12:28:33.851869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 826/1520] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:33.874782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:33.875783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:33.882964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.883415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.884206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:33.884632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:33.914752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-adce1a48-bf36-4818-98e1-63e7ebdb1956 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044087 May 19 12:28:33.915319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 695/1521] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:34.105078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:34.106029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:34.112604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.113098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.113838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.114260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.144667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f0090b62-82e4-48e1-8fce-7767c091f44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043618 May 19 12:28:34.145237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 827/1522] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:34.872424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:34.873550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:34.883755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.884196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.884992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.885427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.930088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:34.931297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:34.937940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.938374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.939126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:34.939547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:34.952630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:34.961067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-32d9ef63-10a3-4a36-b3e0-5bb43374f34f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094969 May 19 12:28:34.961805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 696/1523] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:34.969782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d4898108-4089-49cf-8648-63dd655aaffd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046052 May 19 12:28:34.970345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 828/1524] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:35.162705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:35.164137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:35.170662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.171106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:35.171857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.172281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:35.201379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-36c49572-684b-4e83-a86f-d5ca4706488b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045580 May 19 12:28:35.201938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 697/1525] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:35.982125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:35.983165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:35.987098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:35.988575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:35.994076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.994529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:35.995316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.995748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:35.997012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.997453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:35.998343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:35.998634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:36.030939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d5105d79-2196-4d61-8033-db21cad0f3a0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050441 May 19 12:28:36.031521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 698/1526] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:36.058732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:36.066674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3633a75d-baf4-468a-9661-53c3d5ecdbcc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091065 May 19 12:28:36.067298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 829/1527] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:36.216932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:36.218034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:36.226849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:36.227342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:36.228187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:36.228750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:36.261028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c0f5e02-16e9-4c10-8c70-2e1ad7eb6262 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049165 May 19 12:28:36.261510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 699/1528] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:37.044085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:37.045010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:37.051983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.052425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.053214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.053642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.057332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:37.083424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:37.084009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a55588aa-5de5-4483-b725-53afd428afeb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044825 May 19 12:28:37.084124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:37.084228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 830/1529] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:37.092687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.093274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.094187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.094635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.161764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:37.169807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e57f75ae-2bb5-4150-8c3b-af379048db4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090213 May 19 12:28:37.170374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 700/1530] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:37.277004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:37.278519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:37.288861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.289564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.290827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:37.291528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:37.320972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e8608c42-8b89-4122-bb90-20d683d4bbbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050648 May 19 12:28:37.321525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 831/1531] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:37 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:38.099222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:38.100707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:38.109075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.109529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.110298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.110730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.114750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:38.141690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e66c2861-415e-4698-ac4d-a580dd92b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049328 May 19 12:28:38.142255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 701/1532] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:38.189436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:38.190430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:38.200487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.201262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.202483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.203179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.278700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:38.290694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd20a6aa-43a9-4955-bf6c-7defce1f91ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.107713 May 19 12:28:38.291326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 832/1533] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:38.338000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:38.339037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:38.345484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.345923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.346668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:38.347088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:38.376503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-be62b21b-a8f2-4a8b-9071-071e6ee9fc73 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041967 May 19 12:28:38.377026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 702/1534] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:39.158111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:39.159673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:39.170736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.171465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.172712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.173457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.206705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77f57e62-a804-4578-a136-830fba54e845 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055451 May 19 12:28:39.207327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 833/1535] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:39.312269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:39.312969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:39.320814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.321301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.322121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.322593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.387006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:39.388511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:39.389442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:39.395511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.395716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e4d656b2-23fc-442c-98c0-c007cc442d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087623 May 19 12:28:39.395929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.396046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 703/1536] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:39.396719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:39.397141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:39.426052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f92ddf64-fd96-4bea-9453-d6169f5b166c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041426 May 19 12:28:39.426517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 834/1537] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:28:40.222347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:40.223818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:40.233528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.234197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.235551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.236062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.265755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-758508de-cb01-46b9-8542-2b8eb2d65ef3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050011 May 19 12:28:40.266249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 704/1538] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:40 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:40.416616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:40.417663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:40.428682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.429426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.430689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.431451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.438945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:40.439992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:40.446909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.447349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.448115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:40.448539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:40.479582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-95ac88be-9d31-4081-9120-381671633474 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044392 May 19 12:28:40.480048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 705/1539] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:40.508098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:40.515872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-53abee60-949a-4e73-920a-f04449d1e760 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.105712 May 19 12:28:40.516495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 835/1540] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:41.280090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:41.281053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:41.288093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.288581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.289460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.289934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.321493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8d72574b-f92a-4bba-a325-d15e67d37586 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046125 May 19 12:28:41.322054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 706/1541] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:41.493088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:41.494227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:41.502804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.503274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.504221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.504788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.532839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:41.533350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:41.538781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f4ec5a9f-2283-4bbc-8fb4-e3364ab5fb6d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050615 May 19 12:28:41.539349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 836/1542] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:41.539876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.540316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.541107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:41.541539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:41.603177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:41.611053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e769629-0874-4b03-bf76-9b66bf180df7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.082190 May 19 12:28:41.611620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 707/1543] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:28:42.333503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:42.334446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:42.340545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.340988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.341755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.342151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.372279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d732a5b0-0685-47ff-a894-6d826e0f0782 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042887 May 19 12:28:42.373135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 837/1544] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:42.555023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:42.556508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:42.564748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.565199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.565957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.566392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.596272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e88faf84-931b-4363-8756-a40505eba592 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048039 May 19 12:28:42.596875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 708/1545] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:42.629019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:42.629607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:42.636816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.637268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.638021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:42.638451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:42.706113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:42.713932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e1ba8401-ba4a-4eef-8a96-d2f0dfe4e366 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089350 May 19 12:28:42.714496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 838/1546] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:43.387766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:43.390048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:43.400338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.401112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.402359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.403076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.440336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-37d8dfc1-5d9b-4d47-a426-067543a14e43 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059255 May 19 12:28:43.440935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 709/1547] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:43.612615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:43.614134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:43.623753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.624193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.625062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.625490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.654599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ee4eff5-35db-4dd8-b19e-8d9ede630149 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048635 May 19 12:28:43.655204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 839/1548] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:43.734003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:43.734582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:43.741574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.742020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.742774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:43.743200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:43.807994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:43.816254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-897f7b44-8b12-41b7-b422-49b2a297586a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087679 May 19 12:28:43.816929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 710/1549] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:44.451839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:44.453157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:44.459671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.460110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.460906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.461332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.494541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-854b1f39-8360-4423-9c91-0fdb5c3be7d7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046696 May 19 12:28:44.495140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 840/1550] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:44.670313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:44.671452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:44.679367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.679925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.680910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.681446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.714625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4b249dd9-18d6-4a7d-9a60-d03fa6616ebd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050797 May 19 12:28:44.715104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 711/1551] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:44.836955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:44.838001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:44.846042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.846531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.847288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:44.847716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:44.916298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:44.924302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-68dff466-28dd-4707-a3cb-e628e4954501 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093847 May 19 12:28:44.924946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 841/1552] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:45.511432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:45.513663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:45.520850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.521310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:45.522111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.522535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:45.552562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a7418394-0c89-4a9a-bf66-d39e3c19e7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048300 May 19 12:28:45.553170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 712/1553] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:45.730614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:45.731926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:45.742241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.742972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:45.744213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.744988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:45.775110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d2fbbdf8-91c7-4dfa-8c58-970d92e6899c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051239 May 19 12:28:45.775657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 842/1554] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:45.829106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:28:45.829395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:28:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:28:45.945051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:45.946040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:45.954289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.954724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:45.955506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:45.955920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:46.027087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:46.035712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7eee9f0c-0223-49ce-87e7-8c9141b30b1a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097057 May 19 12:28:46.036283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 713/1555] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:46.569671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:46.571671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:46.582794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:46.583527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:46.584847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:46.585555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:46.618901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c78b1702-81a5-404a-ad3a-854e38255d2e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056968 May 19 12:28:46.619450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 843/1556] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:46.792003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:46.793459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:46.801390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:46.801830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:46.802582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:46.803006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:46.834056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6ce2dc52-050f-4714-9904-e58f3892a2d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049500 May 19 12:28:46.834618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 714/1557] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:47.056403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:47.057349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:47.065636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.066071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.066819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.067241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.140836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:47.150256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-da5f6cd8-9efd-4738-9375-5d1d0e88224f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100386 May 19 12:28:47.150859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 844/1558] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:47.635174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:47.636710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:47.644851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.645291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.646056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.646488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.676202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cf6acea7-cbb3-46ce-a138-4dd417871e83 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047782 May 19 12:28:47.676801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 715/1559] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:47.849160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:47.850153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:47.857359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.857809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.858572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:47.859001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:47.890032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-43e08294-499c-4a5d-989b-d7e245c5a23d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046735 May 19 12:28:47.890807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 845/1560] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:48.171692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:48.172524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:48.179724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.180167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.180952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.181380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.252620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:48.260367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-534c1afa-270e-4b24-a7bc-5c4c65323979 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095434 May 19 12:28:48.260985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 716/1561] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:48.692206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:48.693382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:48.701230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.701785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.702727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.703261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.732248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:48.737560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d47f58d-d968-46c1-a77e-5d9639ac164d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050816 May 19 12:28:48.738120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 846/1562] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:48.905296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:48.906774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:48.909708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=88 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:48.918789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.919520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.920813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:48.921522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:48.947540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:48.952319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b5825270-970d-4140-a211-df168ba1a846 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053664 May 19 12:28:48.952899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 717/1563] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:49.281990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:49.282813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:49.289865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.290312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:49.291063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.291494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:49.360220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:49.367916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bd138235-b988-4031-af00-3fdf346b519f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092940 May 19 12:28:49.368476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 847/1564] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:49.754427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:49.755925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:49.766906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.767737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:49.768489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.768941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:49.797900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7ebcc164-cffd-4bb8-a60c-4f1d5043f4d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051000 May 19 12:28:49.798449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 718/1565] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:49.968931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:49.970422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:49.977326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.977766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:49.978515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:49.978939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:50.009096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ae4ee6ca-029e-4155-90ff-b0b9a973af9c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046902 May 19 12:28:50.009579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 848/1566] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:50.389730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:50.390983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:50.398476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:50.398959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:50.399705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:50.400282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:50.464032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:50.475799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-75c22828-91a4-4587-9847-e54fca57be6b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092482 May 19 12:28:50.476568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 719/1567] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:50.815700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:50.816992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:50.825594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:50.826315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:50.827312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:50.827842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:50.865625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c343f55-fea4-48aa-8f45-45ffbb1813d0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056649 May 19 12:28:50.866155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 849/1568] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:51.025354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:51.026965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:51.036366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.036851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.037611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.038041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.069536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d8ef1194-5fcc-4587-8a4c-69df318866d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050768 May 19 12:28:51.070004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 720/1569] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:51.501591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:51.502651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:51.504674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=82 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:51.511231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.511679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.512445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.512944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.575288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:51.583521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1b5224da-b36d-4feb-bebc-0c9a3777f2c3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089070 May 19 12:28:51.584103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 850/1570] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:51.879399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:51.880399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:51.887577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.888065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.888930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:51.889407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:51.919793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d52e6769-b79e-4993-8d6b-9963064e3301 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045148 May 19 12:28:51.920379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 721/1571] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:52.084172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:52.085381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:52.094388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.094940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.095896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.096471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.130699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-87d5c4b0-de38-4801-8c17-b7c752595083 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052612 May 19 12:28:52.131392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 851/1572] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:52.604241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:52.604868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:52.611895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.612376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.613178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.613606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.674495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:52.682339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c3113161-a5e1-47a6-86c6-f1e619facd3f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.082006 May 19 12:28:52.682854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 722/1573] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:52.935997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:52.937517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:52.945363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.945805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.946602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:52.947092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:52.977113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-639974be-0590-4cfd-9770-38e1e433b359 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047671 May 19 12:28:52.977674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 852/1574] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:53.147748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:53.149272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:53.159556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:53.160002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:53.160796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:53.161228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:53.190148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6fb1d271-251c-4ae1-8a12-d1fc851dfb75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049087 May 19 12:28:53.190680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 723/1575] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:53.591823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:28:53.592032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:28:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:28:53.709158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:53.710115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:53.721259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:53.722080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:53.722853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:53.723277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:53.786220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:53.794508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ff8b2f43-614d-4acc-8f54-2c893fe5ca74 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091970 May 19 12:28:53.795086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 853/1576] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:53.994171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:53.995645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:54.005070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.005521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.006273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.006702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.037847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-79345eb8-f936-4084-ade8-2b6fa96ef1ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051006 May 19 12:28:54.038476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 724/1577] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:54.093562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:28:54.093807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:28:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:28:54.206393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:54.207318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:54.213816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.214267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.215024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.215451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.245869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8d85ca79-85df-4756-a276-1a65dc9f1e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046314 May 19 12:28:54.246295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 854/1578] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:54.820989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:54.821997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:54.831904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.832338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.833129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:54.833553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:54.906269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:54.914148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1376637a-d6ba-4c75-b84a-2d964ff9ea28 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099737 May 19 12:28:54.914712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 725/1579] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:55.052561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:55.053558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:55.060796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.061242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:55.062000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.062455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:55.093038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-692679b2-fa6f-40d2-a3ff-11e82799e0ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045795 May 19 12:28:55.093541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 855/1580] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:55.261486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:55.262616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:55.270434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.271137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:55.272119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.272541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:55.306382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dac6e175-fe06-44db-9ec1-e7488f96b5dc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051600 May 19 12:28:55.306945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 726/1581] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:55.936211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:55.937258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:55.949717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.950270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:55.951250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:55.951929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:56.038760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:56.048771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ca0222d6-eaf4-46af-bc05-52fc28d54f22 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.120683 May 19 12:28:56.049611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 856/1582] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:55 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:56.106881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:56.108207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:56.117085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:56.117688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:56.118711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:56.119290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:56.151951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d82991ec-dbb2-4223-aa10-42484a31bb35 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050385 May 19 12:28:56.152519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 727/1583] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:56.322433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:56.324026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:56.334546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:56.334958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:56.336031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:56.336457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:56.375977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c41f139-9f46-4fe1-a042-7be0387621b6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059802 May 19 12:28:56.376531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 857/1584] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:57.067539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:57.068219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:57.075756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.076214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.077044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.077485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.150920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:57.158896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5ccb9638-327b-4c96-b807-0efb33e2e35d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095652 May 19 12:28:57.159446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 728/1585] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:57.163856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:57.164817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:57.171327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.171795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.172573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.173034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.202579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4cf9512f-c381-4ea6-9968-af43c3f7ec99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042772 May 19 12:28:57.203113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 858/1586] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:57.393638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:57.394975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:57.402654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.403098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.403846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:57.404275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:57.438208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-010b496f-e12a-4447-93bc-697922aac1e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051055 May 19 12:28:57.438948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 729/1587] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:58.180021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:58.180769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:58.189041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.189554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.190444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.190949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.215559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:58.216465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:58.223807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.224273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.225127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.225543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.257013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cd24e186-8eff-4b96-9f86-d64ea4a0365e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045592 May 19 12:28:58.257388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:58.257560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 730/1588] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:28:58.265492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-754eda2d-e94c-4dae-a981-b72787f7b0ca tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091956 May 19 12:28:58.266047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 859/1589] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:58.454996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:58.455860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:58.462534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.462954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.463727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:58.464499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:58.495916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ea69d985-ea26-42d1-8572-d62a43729198 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045798 May 19 12:28:58.496476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 731/1590] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:59.273442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:59.274923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:59.286058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.286323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:59.286947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.287288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:59.288021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.288750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.295423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.296047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.296838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.297262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.319667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aba58f44-6b6d-495c-a028-dda7bf9c5ae7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052896 May 19 12:28:59.320480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 860/1591] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:28:59.359043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:28:59.366980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c0c17fce-cb59-4dbb-9a49-547a47f1c77d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087021 May 19 12:28:59.367536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 732/1592] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:28:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:28:59.513877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:28:59.514796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:28:59.521240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.521681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.522425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:28:59.522849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:28:59.552895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a217cb9a-1267-41e9-918e-da1795f688be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043465 May 19 12:28:59.553438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 861/1593] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:28:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:00.335655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:00.337192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:00.344279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.344740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.345489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.345913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.375816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5e7ee942-744a-422c-84a2-d24628885fc8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046514 May 19 12:29:00.376357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 733/1594] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:00.383331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:00.383936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:00.391235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.391672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.392419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.392866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.455335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:00.463220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e2a8aa9-ab44-4ca8-890a-013b87b32751 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083632 May 19 12:29:00.463776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 862/1595] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:00.569537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:00.571067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:00.581473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.582213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.583466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:00.584186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:00.603187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:00.617159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fc0523c2-d8c5-4e79-a9b2-59274853b34c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054757 May 19 12:29:00.617714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 734/1596] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:01.391961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:01.393530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:01.403410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.403857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.404612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.405072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.421631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:01.436460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c72376a-f9da-4553-9471-f48ba3c7058f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051193 May 19 12:29:01.437227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 863/1597] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:01.483872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:01.484497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:01.491374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.491836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.492587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.493042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.555521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:01.563635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4b0001cf-e254-447e-9307-42e2a5ee1390 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084064 May 19 12:29:01.564317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 735/1598] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:01.633597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:01.635356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:01.642211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.642582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.643327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:01.643750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:01.674613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-beaf9a4c-7c48-43c6-b83b-372ce144ed3f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047627 May 19 12:29:01.675161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 864/1599] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:02.452548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:02.453801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:02.462156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.462741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.463731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.464488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.502011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9918fac3-c5c7-4c1b-a3af-641c96ba8ba3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055284 May 19 12:29:02.502700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 736/1600] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:02.584290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:02.585334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:02.593659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.594098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.594843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.595264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.687154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:02.688088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:02.695306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.695767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.696567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:02.697038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:02.737246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d011826b-6815-4ee8-af1d-83a607a1261e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054065 May 19 12:29:02.737767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 737/1601] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:02.829874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:02.838033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a24a0d97-c91f-4c24-b67d-5ac429872811 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.260056 May 19 12:29:02.838596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 865/1602] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:03.523077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:03.524548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:03.532247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.532734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.533484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.533914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.564311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7bc9bf04-d612-4dae-9d4e-e9b5ecc23d64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047957 May 19 12:29:03.564888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 738/1603] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:03.753721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:03.755217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:03.764393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.764797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.765542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.765967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.796580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-627664aa-3d2a-4420-92b8-515cc0296b09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049874 May 19 12:29:03.797213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 866/1604] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:03.854004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:03.854617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:03.860950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.861438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.862194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:03.862627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:03.926085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:03.934020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a25d5870-c902-4b71-8113-cf5d5438c7f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083923 May 19 12:29:03.934590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 739/1605] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:04.578450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:04.579642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:04.587959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.588547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:04.589586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.590218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:04.623675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e89a748-c2ed-4b59-9d2e-c36242ea18e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050970 May 19 12:29:04.624217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 867/1606] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:04.811869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:04.813006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:04.821160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.821689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:04.822572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.823082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:04.856295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e92385dc-6069-4b0d-826a-cffd96810d89 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049987 May 19 12:29:04.856899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 740/1607] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:04.959640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:04.960688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:04.969372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.969808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:04.970560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:04.970982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:05.044013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:05.051766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65e95729-fb67-406b-8b79-14efd285f4a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098591 May 19 12:29:05.052333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 868/1608] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:05.635731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:05.636726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:05.643290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:05.643753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:05.644509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:05.644974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:05.674750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0ec7cf35-c4cc-434f-adbe-a27ff3ad920e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043033 May 19 12:29:05.675313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 741/1609] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:05.874518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:05.876347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:05.886542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:05.887279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:05.888282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:05.888734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:05.917736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4bcad14d-192c-4ff3-ae0f-7737aea0690b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050125 May 19 12:29:05.918268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 869/1610] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:06.068751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:06.069370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:06.076085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.076531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.077277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.077743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.141833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:06.149728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-707320ec-6e46-4846-8f2d-d0945e034741 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085042 May 19 12:29:06.150298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 742/1611] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:06.691216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:06.692775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:06.702755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.703201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.703965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.704395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.734785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3acf0df8-0590-48da-a9ad-42be7feec808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050401 May 19 12:29:06.735333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 870/1612] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:06.776313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:29:06.776470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:29:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:29:06.933738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:06.935203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:06.945463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.946149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.947427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:06.948083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:06.978113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9c40eaec-d9db-4ae8-979b-9b98906d91cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051037 May 19 12:29:06.978681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 743/1613] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:07.167162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:07.167754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:07.174155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:07.174585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:07.175324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:07.175746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:07.237087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:07.245083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a7cac706-5168-471b-8dc8-18eb9a0224ee tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.081846 May 19 12:29:07.245657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 871/1614] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:07.750430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:07.751543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:07.760007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:07.760475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:07.761457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:07.761984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:07.794585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a1dd9d5e-f8d5-4f0e-a8e0-057737303f3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050245 May 19 12:29:07.795136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 744/1615] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:07.993869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:07.995325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:08.002843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.003300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.004065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.004490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.033731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-385156db-d373-4123-afe3-7c006380630e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046487 May 19 12:29:08.034189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 872/1616] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:08.262615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:08.263254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:08.269726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.270163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.270915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.271331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.333321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:08.341287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ecb74c76-7d1c-43b6-8f61-5e4695e1ee8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.082674 May 19 12:29:08.341880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 745/1617] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:08.810368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:08.811294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:08.817990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.818431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.819198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:08.819629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:08.854709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e7f729ff-7b8f-433b-8f0e-41f004448151 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050750 May 19 12:29:08.855263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 873/1618] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:09.051567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:09.053121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:09.064083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.064783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.066034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.066754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.098799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1a1aa1c9-26b6-4e04-9693-6879eb7a4e00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053878 May 19 12:29:09.099355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 746/1619] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:09.362462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:09.363336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:09.373456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.373897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.374652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.375415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.437789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:09.445658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e38247a4-7c95-4766-aebe-3aed6f9a103d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089836 May 19 12:29:09.446218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 874/1620] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:09.871275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:09.872799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:09.883424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.884164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.885470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:09.886183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:09.915699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b56e9f30-09d1-4c63-9fcf-a41a56b0c93e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051188 May 19 12:29:09.916247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 747/1621] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:10.115270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:10.116802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:10.125597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.126015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.126784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.127219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.158873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ff7125c3-5a93-475d-9811-1c81db128400 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050374 May 19 12:29:10.159597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 875/1622] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:10.466811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:10.467690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:10.478533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.481821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.482603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.483038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.554576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:10.562174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0c7226a6-246a-4a8c-b66c-26f7759df1f3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101965 May 19 12:29:10.562747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 748/1623] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:10.931195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:10.932713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:10.940164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.940604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.941371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:10.941790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:10.972222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-279bfcef-c159-471e-9a0e-126a4f58e2f1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047676 May 19 12:29:10.972793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 876/1624] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:11.175192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:11.176315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:11.184595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:11.185119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:11.186055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:11.186594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:11.220993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e3f301d1-49cc-4a58-a3da-1b83f525bf82 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050916 May 19 12:29:11.221559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 749/1625] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:11.364927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:29:11.365186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:29:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:29:11.579684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:11.580292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:11.586847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:11.587289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:11.588038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:11.588463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:11.650400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:11.658634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-671d4492-e69d-4e9b-b618-a0539d1920fb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.082895 May 19 12:29:11.659204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 877/1626] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:11.987887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:11.989169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:11.998756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:11.999380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.000930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:12.001710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.035785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-98ff6722-fa81-42ad-92fc-97b10d66f97d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054812 May 19 12:29:12.036352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 750/1627] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:12.235471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:12.236389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:12.242737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:12.243116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.243872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:12.244300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.274042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f03c28c-1e2d-4d15-88d7-79eb9af896b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042600 May 19 12:29:12.274590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 878/1628] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:12.679937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:12.680990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:12.691339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:12.692092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.693383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:12.694087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:12.700483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:12.758375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:12.766776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a33ab411-7b75-4afc-aae6-433d4ff0c97a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093067 May 19 12:29:12.767293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 751/1629] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:13.052437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:13.053905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:13.062607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.063050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.063808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.064242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.093862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e369fcf-858e-49e8-afee-f3d3fb0c8f69 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048510 May 19 12:29:13.094421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 879/1630] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:13.290369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:13.291857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:13.302895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.303533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.304826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.305542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.336329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5d472403-bf25-4128-be07-5f38eaf7b2e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052676 May 19 12:29:13.336910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 752/1631] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:13.787856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:13.788957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:13.799895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.800627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.802011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:13.802732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:13.809360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:13.872065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:13.880692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8cec8209-4355-4b22-a9b2-b2f07cc74af0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099291 May 19 12:29:13.881291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 880/1632] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:14.111757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:14.113282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:14.123855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.124630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.126353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.127085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.161712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ff29fe46-4957-4e19-b4c8-3cf7c8551d3f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056693 May 19 12:29:14.162279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 753/1633] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:14.351133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:14.352504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:14.361072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.361514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.362330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.362692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.392665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5cec9a31-4cdc-46ae-b58c-cd755461d5bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047687 May 19 12:29:14.393290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 881/1634] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:14.900295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:14.901261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:14.911357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.912082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.913201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:14.914041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:14.980730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:14.988815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-378bb34a-15dd-485b-abd4-10e61b600a5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093763 May 19 12:29:14.989390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 754/1635] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:15.177156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:15.178472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:15.188889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:15.189343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:15.190101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:15.190534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:15.221001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f790ce4-3901-41a7-90df-125aaedc2d03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050632 May 19 12:29:15.221564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 882/1636] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:15.407873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:15.408827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:15.415058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:15.415542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:15.416358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:15.416811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:15.446887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-edc35406-f172-48d8-973f-f5a1be683bb1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043917 May 19 12:29:15.447373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 755/1637] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:16.010291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:16.011279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:16.021442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.022175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.023414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.024152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.092057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:16.099910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ace1c4d5-7498-4b34-9e2a-c512435246db tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096158 May 19 12:29:16.100467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 883/1638] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:16 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:16.237566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:16.239033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:16.250430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.250869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.251616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.252038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.281223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ad8c2f06-eba4-416c-a1c6-db7b3d611e21 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050231 May 19 12:29:16.281715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 756/1639] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:16.463009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:16.464523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:16.477432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.478183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.479472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:16.480182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:16.512366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2bffd113-a60d-4106-bf7a-32b5315f19f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056466 May 19 12:29:16.512966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 884/1640] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:17.121499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:17.122486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:17.133321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.134059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.135365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.136068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.210145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:17.218470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-966834d0-fb5d-4c7b-b982-45e7e9a8ee1b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.104192 May 19 12:29:17.219076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 757/1641] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:17.297033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:17.298003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:17.304677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.305153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.305942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.306385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.336744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5e66146-7199-4a13-91cd-2914608b15aa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046444 May 19 12:29:17.337288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 885/1642] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:17.527852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:17.529377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:17.537048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.537490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.538370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:17.538647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:17.568687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fffa1fae-87c2-47fe-aa50-95f2c146b656 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047584 May 19 12:29:17.569235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 758/1643] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:18.239570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:18.240539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:18.251271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.251706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.252444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.252898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.325244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:18.333177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-78593434-d53b-4e38-9d44-4e073e7ed1aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100256 May 19 12:29:18.333777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 886/1644] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:18.347659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:18.348612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:18.355180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.355634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.356372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.356834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.388183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1392141f-28f1-42fa-96d0-dd75ff57141a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044617 May 19 12:29:18.388731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 759/1645] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:18.584595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:18.586289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:18.596526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.597290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.598545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:18.599243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:18.632059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90582bd3-dd78-4f57-8e14-e87ea79b99e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054261 May 19 12:29:18.632609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 887/1646] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:19.353878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:19.354811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:19.358471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=86 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:19.364061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.364498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.365272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.365694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.399615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:19.400655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:19.408692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.409158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.409928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.410351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.427723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:19.435643 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9dcd2f74-4e64-447c-88b7-064f5354bc16 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088156 May 19 12:29:19.436186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 760/1647] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:19.441729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3f317740-5126-4af1-be11-e1659b8d8acd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046103 May 19 12:29:19.442296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 888/1648] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:19.649109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:19.650458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:19.659264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.659714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.660471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:19.660905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:19.689971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-47731ef3-aa0e-46ae-8dff-a9927dd946b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047566 May 19 12:29:19.690536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 761/1649] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:20.457226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:20.458349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:20.458998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:20.459165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:20.469118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.469312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.469559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.469714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.470323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.470478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.470748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.470915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.500684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c7e642a3-dfaf-42ce-b4ba-34370b590a3b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049865 May 19 12:29:20.501640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 889/1650] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:20.534165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:20.542041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-80e04ec8-5899-4f1c-8ffa-2a540119fc71 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091348 May 19 12:29:20.542606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 762/1651] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:20.704714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:20.705917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:20.713908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.714279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.715022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:20.715436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:20.745182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14c06629-9b3b-46ca-88d5-ba02026ffcb8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046236 May 19 12:29:20.745723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 890/1652] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:21.512508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:21.513486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:21.520471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.520938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.521689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.522147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.552035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-192ca29d-0463-4628-b468-35be2813eeaa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043469 May 19 12:29:21.552620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 763/1653] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:21.563402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:21.563948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:21.566443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=82 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:21.572052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.572493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.573301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.573736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.637708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:21.645739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e7fbb88-b8ab-4286-a932-20cb02dd36ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086174 May 19 12:29:21.646270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 891/1654] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:21.760939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:21.762411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:21.770009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.770518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.771266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:21.771694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:21.801563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1df0cf2b-0031-4fa6-af04-8dfc6036c6d1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047221 May 19 12:29:21.802111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 764/1655] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:22.567740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:22.569057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:22.578985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.579483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.580244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.580693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.610299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-61fe4bb3-8a21-41ee-932d-54b027b73cf3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049186 May 19 12:29:22.610884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 892/1656] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:22.666197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:22.667132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:22.677755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.678443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.679608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.680284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.764536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:22.772147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b92f028f-2114-4ad4-b0d2-dafdee297681 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.112420 May 19 12:29:22.772738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 765/1657] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:22.813709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:22.814672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:22.821170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.821615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.822368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:22.822788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:22.855014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-617b7c5f-2e8f-4cc3-b30e-f7f03b14a182 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045435 May 19 12:29:22.855593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 893/1658] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:23.624150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:23.625602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:23.632057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.632504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.633294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.633722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.663924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-007a0070-8c4c-40ab-bbea-3eba3127946e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045009 May 19 12:29:23.664482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 766/1659] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:23.793469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:23.794288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:23.802155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.802597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.803349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.803774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.867531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:23.868442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:23.868891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:23.875730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.876187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.876928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:23.877073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b2fd7cc1-aaba-47af-91f3-f21b15ec63a1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090388 May 19 12:29:23.877354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:23.877669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 894/1660] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:23.903839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:23.908863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d899c6d3-64d9-4c7f-a90c-6b36da8bb81c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045710 May 19 12:29:23.909417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 767/1661] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:24.678034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:24.679154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:24.687188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.687737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.688692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.689240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.721585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6b8ab781-ed65-4f33-9c2e-3d9f0c30e0f8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048619 May 19 12:29:24.722146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 895/1662] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:24.898329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:24.898930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:24.909012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.909495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.910325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.910802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.920794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:24.922295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:24.929416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.930072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.930861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:24.931296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:24.957587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:24.962474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ba5c1bda-cc46-4d36-ba00-d796c98778db tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045711 May 19 12:29:24.963032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 896/1663] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:24.974722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:24.982387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a7f60023-dfe6-466a-848b-cd0c8a78fdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090514 May 19 12:29:24.982953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 768/1664] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:25.736458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:25.737700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:25.747010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:25.747692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:25.748799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:25.749904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:25.781401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3f0b2861-e2be-4c9a-b5b7-3e1b2e14d3a7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050728 May 19 12:29:25.781966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 897/1665] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:25.978290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:25.979811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:25.988983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:25.989360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:25.990122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:25.990555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:26.000074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:26.000821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:26.007806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:26.008298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:26.009174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:26.009656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:26.021529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-48cc893c-2630-413a-b309-271d0442af4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049804 May 19 12:29:26.022104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 769/1666] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:26.071234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:26.078837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ab5e9f2-3b24-426d-8867-a15a8d8ab708 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083227 May 19 12:29:26.079393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 898/1667] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:26.794305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:26.795228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:26.801950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:26.802390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:26.803146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:26.803574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:26.834145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-504db770-7e00-41e3-9ff5-ce31bd06d452 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044270 May 19 12:29:26.834691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 770/1668] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:27.037671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:27.039211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:27.049126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.049499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.050373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.050802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.080293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-baca3e01-ddf2-4d6f-9fb4-f21d093accad tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049350 May 19 12:29:27.080942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 899/1669] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:27.098891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:27.099877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:27.109846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.110288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.111048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.111477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.178168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:27.185650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e09a69bf-cb8a-431c-8491-bcdf4d468b8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093023 May 19 12:29:27.186218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 771/1670] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:27.847336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:27.848251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:27.854831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.855270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.856017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:27.856441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:27.886095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6f68fc12-c0ba-4595-8bff-12da731eba7f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043327 May 19 12:29:27.886742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 900/1671] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:28.095538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:28.096598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:28.104660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.105350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.105971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.106467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.139112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6cea5b81-486f-4747-93a0-89e9ab6e8369 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050150 May 19 12:29:28.139677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 772/1672] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:28.206971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:28.207980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:28.219283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.220068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.220885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.221301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.284366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:28.292148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-641646ca-6687-47f6-b4f9-73321b629297 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091930 May 19 12:29:28.292716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 901/1673] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:28.905125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:28.906624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:28.913855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.914296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.915054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:28.915480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:28.945010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f9165ab4-5e9e-421b-866b-72d735c29e26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046649 May 19 12:29:28.945567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 773/1674] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:29:29.154375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:29.155876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:29.165167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.165812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:29.166966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.167661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:29.208561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3eddef66-7013-4227-a9d5-2cd3c6c8c6e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060750 May 19 12:29:29.209229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 902/1675] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:29.312332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:29.313376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:29.324232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.325236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:29.326489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.327185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:29.395805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:29.407590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-963294cb-4064-4f53-af62-1edf29d03833 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101117 May 19 12:29:29.408089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 774/1676] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:29.960445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:29.961957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:29.973204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.973928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:29.975155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:29.975855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:30.009447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5c1f2529-1b67-446c-85f2-8e4385e3af8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055748 May 19 12:29:30.009999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 903/1677] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:30.222820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:30.223780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:30.231484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:30.232002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:30.232918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:30.233445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:30.265847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cef4ce6f-3525-46be-82a5-7afb1ef29488 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048790 May 19 12:29:30.266392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 775/1678] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:30.428200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:30.429299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:30.439562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:30.440047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:30.440831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:30.441259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:30.516796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:30.525042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8795862a-7de9-443b-b49c-55e435e0e19e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.103293 May 19 12:29:30.525632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 904/1679] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:31.025486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:31.026955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:31.035799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.036238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.037020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.037442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.066326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d54672ba-d6cf-4fd5-8ff0-22ceec1b66e3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047973 May 19 12:29:31.066866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 776/1680] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:31.282328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:31.283671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:31.294509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.295274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.296472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.297236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.326595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-36e2d583-5ea3-445b-90c5-72ce05ca60da tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051100 May 19 12:29:31.327140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 905/1681] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:31.545972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:31.546686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:31.553396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.553834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.554963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:31.555404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:31.619856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:31.628078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d221aebb-59c7-4975-8ce9-bbf68e9136a8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088318 May 19 12:29:31.628654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 777/1682] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:32.080378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:32.081331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:32.087610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.088093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.088902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.089339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.119259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be5c1835-6c66-44e5-8b6d-ffe88968d0de tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044826 May 19 12:29:32.119809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 906/1683] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:32.342772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:32.344134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:32.351787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.352236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.353480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.384142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d81c0fa4-12c5-4bc3-ac76-ff31d7a720fc tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048254 May 19 12:29:32.384729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 778/1684] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:32.649267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:32.650138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:32.660044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.660489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.661282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:32.661709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:32.730172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:32.738158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-81a7acfd-3154-4858-a149-005132f033c1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095555 May 19 12:29:32.738730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 907/1685] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:33.135113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:33.136569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:33.146898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.147628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.148915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.149615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.181762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cfcbe883-c10c-4ada-bf10-fa47fb6c0b22 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053428 May 19 12:29:33.182303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 779/1686] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:33.399763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:33.401327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:33.410145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.410586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.411340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.411763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.442654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-614277ff-ad96-4d5c-8ae9-e57a944e8c7e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049618 May 19 12:29:33.443214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 908/1687] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:33.759541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:33.760391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:33.769913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.770358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.771123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:33.771554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:33.844311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:33.854078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-705999b9-0be7-483a-b991-55034bca034f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101237 May 19 12:29:33.854652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 780/1688] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:34.197945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:34.199414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:34.208218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.208631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.209405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.209828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.240011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-485fee47-d586-4e90-bf0e-1d9a308e2c52 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048710 May 19 12:29:34.240538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 909/1689] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:34.461233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:34.462802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:34.473235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.473972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.475152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.475893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.510397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a92842f-5b95-4df1-aac6-5179478bf2e4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056058 May 19 12:29:34.510963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 781/1690] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:34.879540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:34.880254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:34.888150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.888703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.889654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:34.890176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:34.955011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:34.963276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a18804a-1b02-42c8-80cc-8356188af30b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090203 May 19 12:29:34.963849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 910/1691] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:35.256416 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:35.257782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:35.268543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.269010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:35.269753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.270183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:35.304337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cf25ac7-8f33-4c69-967c-1048c124bbd1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055210 May 19 12:29:35.304932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 782/1692] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:35.525101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:35.526383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:35.535596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.536036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:35.536830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.537251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:35.566516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-016cf656-4266-47bc-b80c-4e1d37b517c2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047725 May 19 12:29:35.567064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 911/1693] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:35.983402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:35.984399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:35.991808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.992284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:35.993074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:35.993472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:36.056168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:36.063884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0988c414-7e0b-4869-8621-022d6ad3f8c7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086958 May 19 12:29:36.064465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 783/1694] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:36.320322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:36.321944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:36.332237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:36.333024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:36.334257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:36.334962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:36.368992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d9f3698-9aee-474d-9e31-94a8d0f62769 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055382 May 19 12:29:36.369554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 912/1695] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:36.584191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:36.585620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:36.596721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:36.597501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:36.598744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:36.599454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:36.621023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:36.634789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c19101df-56ea-4687-a432-35c0db96f92e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057347 May 19 12:29:36.635333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 784/1696] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:37.084942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:37.085946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:37.096831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.097276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.098036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.098465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.120893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:37.169495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:37.177399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9b409f29-47ec-4096-8f51-6bc72395cdbd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098978 May 19 12:29:37.177966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 913/1697] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:37.385253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:37.386603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:37.395617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.396060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.396853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.397276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.426661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e9dda411-6a86-42b4-bfdf-f7763037513a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048179 May 19 12:29:37.427234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 785/1698] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:37.650848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:37.652340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:37.661517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.661956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.662703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:37.663128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:37.692802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cff128d3-afad-4a78-b093-864d51decb36 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048637 May 19 12:29:37.693350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 914/1699] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:37 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:38.198636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:38.199469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:38.208835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.209279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.210037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.210464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.271002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:38.278656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a003e9c2-fad7-4d4a-845a-dfa7f12b1040 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086814 May 19 12:29:38.279206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 786/1700] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:38.441742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:29:38.442691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:38.449447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.449868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.450623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.451049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.555384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7f573b54-455c-440d-b1a6-c2aef974a9da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action" status: 200 len: 76565 microversion: 2.1 time: 0.119049 May 19 12:29:38.556296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 915/1701] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:29:38 2022] POST /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121/action => generated 76565 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:29:38.708592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:38.709563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:38.716543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.717057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.717881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:38.718350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:38.749663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ff514927-7b1a-46cd-92fd-017eaaf888c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046654 May 19 12:29:38.750209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 787/1702] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:39.300919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:39.302036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:39.314065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:39.314834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:39.315748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:39.316170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:39.379320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:39.387832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-613cf5d3-186f-479e-afa1-efff6975439e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094129 May 19 12:29:39.388396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 916/1703] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:39.765418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:39.766880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:39.774304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:39.774757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:39.775511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:39.775933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:39.808135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-50bdbded-cfbc-4205-8a2e-92767d781d14 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049237 May 19 12:29:39.809035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 788/1704] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:40.408868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:40.409832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:40.421093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:40.421887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:40.423131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:40.423801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:40.487766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:40.495564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-471ee0f5-ff39-49f0-ba4d-e7fb43921650 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093188 May 19 12:29:40.496131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 917/1705] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:40.823820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:40.825346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:40.837218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:40.837957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:40.839212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:40.839928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:40.886162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9440878f-baa4-430a-a1b2-0dc224119000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068842 May 19 12:29:40.886697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 789/1706] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:41.512914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:41.513522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:41.520591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:41.521060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:41.521813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:41.522230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:41.584505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:41.592501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-086e5175-3cf3-4bb5-a728-2e808c0ffc97 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083470 May 19 12:29:41.593107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 918/1707] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:41.898173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:41.899134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:41.906028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:41.906478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:41.907306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:41.907734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:41.942454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0f2c3aeb-1c33-4d07-852f-3533b0d7a969 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048012 May 19 12:29:41.943265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 790/1708] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:42.611834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:42.612674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:42.621827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:42.622413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:42.623404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:42.624299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:42.690323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:42.698450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ad6864f4-3bc7-4734-a62c-a508de9dcbc8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092003 May 19 12:29:42.698981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 919/1709] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:42.959091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:42.960619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:42.971433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:42.972267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:42.973313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:42.973741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:43.005116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afdaf3e5-9e5f-49f2-8034-bb08a6a05603 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053012 May 19 12:29:43.005693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 791/1710] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:43.722326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:43.723308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:43.730276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:43.730715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:43.731461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:43.731881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:43.806842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:43.814418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77602cfa-2699-4927-a05a-1b6bdee93cfb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099023 May 19 12:29:43.814985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 920/1711] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:44.020574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:44.021600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:44.028239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:44.028728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:44.029493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:44.029929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:44.059728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d5daf84e-8a8a-4f15-a9fb-36b60bdd9381 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045860 May 19 12:29:44.060279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 792/1712] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:44.834713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:44.835346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:44.842844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:44.843315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:44.844251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:44.844736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:44.919695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:44.929995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a767ffab-4179-40ed-bef5-b7cff62ec2d4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100585 May 19 12:29:44.930562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 921/1713] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:45.073751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:45.074784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:45.084849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:45.085333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:45.086466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:45.086982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:45.120240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-192a0e9d-c34a-45f9-aa7f-55b546f5bb04 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051385 May 19 12:29:45.120832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 793/1714] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:45.951447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:45.952364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:45.963554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:45.964282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:45.965599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:45.966352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:46.036252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:46.044356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9fbb2c9f-f434-45c9-a300-5ced98cbf143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099408 May 19 12:29:46.044957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 922/1715] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:46.132726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:46.133746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:46.141172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:46.141693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:46.142494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:46.142975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:46.174044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ee999e73-6c50-41a5-8a06-f1b6c16782e7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045943 May 19 12:29:46.174637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 794/1716] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:47.065619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:47.066645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:47.077746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:47.078510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:47.079794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:47.080440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:47.158258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:47.169319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f79e919a-fd05-40e8-8ae5-4f7b8832a6e7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.109761 May 19 12:29:47.170039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 923/1717] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:47.185511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:47.186419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:47.193122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:47.193568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:47.194320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:47.194744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:47.230275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7d345786-3453-4fa3-99db-2413999dd7c7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048398 May 19 12:29:47.230948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 795/1718] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:48.192348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:48.193007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:48.199914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:48.200359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:48.201151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:48.201578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:48.242448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:48.243429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:48.250295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:48.251156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:48.251588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:48.251993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:48.267913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:48.275494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-07c560bd-c941-4eae-9db5-bb9eacbadb56 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089454 May 19 12:29:48.276054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 924/1719] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:48.283709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2f5503cc-859c-4543-a96d-ce5032e504ba tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045537 May 19 12:29:48.284266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 796/1720] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:49.291734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:49.292413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:49.295030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:49.296380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:49.299540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:49.300058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:49.300916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:49.301579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:49.302766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:49.303218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:49.303980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:49.304661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:49.335241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba0eff5f-83c7-43b8-8c85-39915afe59c8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044071 May 19 12:29:49.335822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 797/1721] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:49.365441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:49.373155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-103d45bc-f749-4fc2-8fa4-5592dbb6da4a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085555 May 19 12:29:49.374051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 925/1722] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:50.350805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:50.351877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:50.354666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=76 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:50.361565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:50.362085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:50.362996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:50.363498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:50.390729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:50.391381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:50.395692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d2677725-69ed-412e-96f1-0df5394eb53b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051261 May 19 12:29:50.396261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 798/1723] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:50.398089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:50.398505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:50.399295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:50.399721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:50.473433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:50.481478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e357ab01-d1aa-4353-aec4-7afc67ebe78a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094660 May 19 12:29:50.482046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 926/1724] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:51.412154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:51.413691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:51.424562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:51.425350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:51.426600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:51.427319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:51.462572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aa273513-ad58-4591-9a99-c1182d12ad1e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057204 May 19 12:29:51.463186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 799/1725] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:51.498038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:51.498974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:51.505851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:51.506293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:51.507101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:51.507535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:51.569277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:51.581216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4765fa0f-915b-4e3d-b382-3b8393ff9c98 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086812 May 19 12:29:51.582032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 927/1726] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:52.477282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:52.478112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:52.485149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:52.485618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:52.486372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:52.486802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:52.517409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab006bad-b4b3-463f-9ef7-8126d085e6b3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044717 May 19 12:29:52.517976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 800/1727] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:52.602894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:52.603874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:52.607783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=74 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:52.616937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:52.617669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:52.618919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:52.619622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:52.688963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:52.697131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d04d5e50-d455-44e5-a026-a5fa7247f420 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100829 May 19 12:29:52.697708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 928/1728] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:53.529615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:53.530533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:53.537501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:53.537946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:53.538699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:53.539120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:53.542756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:53.569170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-47511b68-21ff-4c72-8ad2-b83b4fe2de0f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043590 May 19 12:29:53.569979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 801/1729] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:53.718382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:53.719411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:53.730450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:53.731193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:53.732428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:53.733188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:53.799139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:53.806980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-34fde610-83ef-46e7-849d-a5edfdcd7a24 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094944 May 19 12:29:53.807545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 929/1730] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:54.585335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:54.586466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:54.595265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:54.595825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:54.596811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:54.597373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:54.631290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0d56330a-6f08-4a20-86f1-575f4a0dcd35 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052442 May 19 12:29:54.631858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 802/1731] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:54.825157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:54.826062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:54.834160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:54.834680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:54.835573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:54.836079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:54.856346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:54.900282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:54.908420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e2591b30-86d2-4c29-a865-a90ee4ad16e5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088044 May 19 12:29:54.909029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 930/1732] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:55.643353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:55.644302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:55.651421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:55.651871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:55.652666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:55.653100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:55.684359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b5a448b6-a366-44d7-ba6b-03f2e3c5d66f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044970 May 19 12:29:55.684984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 803/1733] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:55.926898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:55.927706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:55.935164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:55.935635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:55.936417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:55.936896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:56.008192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:56.019187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1079c066-a194-4eb1-81fd-a99aec66f9b3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096073 May 19 12:29:56.020105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 931/1734] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:55 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:56.700079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:56.701477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:56.713497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:56.713954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:56.714830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:56.715618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:56.749799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3d49254c-5716-43ac-bddb-5b5247e4745d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055379 May 19 12:29:56.750353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 804/1735] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:57.041958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:57.042932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:57.054392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:57.055130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:57.056393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:57.057348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:57.127325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:57.135197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f27c17c4-3ab4-4701-b32a-fd2451de8143 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100617 May 19 12:29:57.135786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 932/1736] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:57.766335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:57.767705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:57.779193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:57.779943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:57.781339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:57.782064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:57.813610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7d798aef-a2b6-4c8a-b1c5-79c5b8b93e17 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054074 May 19 12:29:57.814165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 805/1737] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:58.156291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:58.157294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:58.168435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:58.169220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:58.170548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:58.171255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:58.238372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:58.246243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4d894eba-1f0e-4df5-b3bb-7a0bb378604c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096535 May 19 12:29:58.246822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 933/1738] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:58.830094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:58.831585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:58.840331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:58.840805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:58.841556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:58.841979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:58.873393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5c0ed6c4-de1f-4e28-af7c-d97f6670dff8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049780 May 19 12:29:58.873969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 806/1739] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:29:59.263980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:59.264588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:59.271379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:59.271817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:59.272588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:59.273094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:59.336260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:29:59.344594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-318bb796-48a9-49ac-bff0-8349c19f208d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084647 May 19 12:29:59.345204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 934/1740] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:29:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:29:59.891021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:29:59.892538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:29:59.901467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:59.901908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:59.902667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:29:59.903094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:29:59.932522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba064b75-1557-4c62-8b0b-2243f040a289 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048310 May 19 12:29:59.933106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 807/1741] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:29:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:00.364698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:00.365727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:00.373598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:00.374045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:00.374829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:00.375256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:00.443222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:00.451205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ff56247c-59d8-43e9-8028-d6b474d6910c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093036 May 19 12:30:00.451802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 935/1742] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:00.948837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:00.950341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:00.961471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:00.962219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:00.963473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:00.964196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:00.998600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d90c386b-d98a-4f50-9f97-e309ee73676a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056293 May 19 12:30:00.999144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 808/1743] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:01.475459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:01.476621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:01.483718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:01.484152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:01.484954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:01.485379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:01.550612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:01.558199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0b450a18-dd7e-4f5c-8fc5-6307cf388732 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089279 May 19 12:30:01.558888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 936/1744] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:02.014248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:02.015709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:02.025530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:02.025972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:02.026718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:02.027147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:02.056204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fd4a646c-73f2-40b3-a93d-7b30f4cb1991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048773 May 19 12:30:02.056769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 809/1745] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:02.583153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:02.584156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:02.591810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:02.592254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:02.593051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:02.593470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:02.654663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:02.662647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7b67e882-dee7-4c9c-87c1-df67dfda42fe tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086014 May 19 12:30:02.663210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 937/1746] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:03.071822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:03.072934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:03.080979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:03.081505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:03.082397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:03.082900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:03.115233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba620d7e-f4de-4376-9b2b-277caeca73be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049614 May 19 12:30:03.115786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 810/1747] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:03.683513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:03.684506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:03.695594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:03.696033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:03.696828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:03.697257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:03.770190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:03.784382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a2e770f-11d5-4097-986e-dfbc62c38228 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.106649 May 19 12:30:03.785716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 938/1748] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:04.131810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:04.133348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:04.144326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:04.145109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:04.146413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:04.147168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:04.177020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f02c4cdf-12fc-4cbe-8b43-ce2ee5aefce6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051898 May 19 12:30:04.177568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 811/1749] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:04.808946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:04.809660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:04.817821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:04.818333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:04.819206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:04.819699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:04.892253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:04.900489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6608a5dc-2c54-4c77-91b5-99d47e59fbd1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096482 May 19 12:30:04.901070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 939/1750] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:05.192976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:05.194055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:05.201407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:05.201846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:05.202590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:05.203012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:05.234215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-36eab4c0-ce7b-4c03-96e7-1a30bffd1b01 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047992 May 19 12:30:05.234777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 812/1751] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:05.921799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:05.922748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:05.933759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:05.934533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:05.935720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:05.936428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:06.007255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:06.015310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-44868d2e-aef3-4a67-8147-cf9b4c204bad tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100079 May 19 12:30:06.015997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 940/1752] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:06.248098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:06.249197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:06.257753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:06.258199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:06.258957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:06.259382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:06.289350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15e4e651-0c22-45b6-875f-5f0d58b11cc3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046653 May 19 12:30:06.289900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 813/1753] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:07.036827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:07.037790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:07.048669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:07.049431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:07.050668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:07.051376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:07.114985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:07.123398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-72c6e458-48f4-4a97-984f-7f728bf1064d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092698 May 19 12:30:07.124289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 941/1754] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:07.305161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:07.305980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:07.312869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:07.313312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:07.314067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:07.314498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:07.344658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3eb66067-3eab-420b-ae1b-9de4e968ae84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046004 May 19 12:30:07.345221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 814/1755] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:08.144719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:08.145702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:08.153582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:08.154025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:08.154786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:08.155221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:08.221120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:08.229438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b76c8a15-a78c-4dd0-8f60-c7dce531dd6e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091170 May 19 12:30:08.230019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 942/1756] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:08.359191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:08.360137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:08.367154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:08.367593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:08.368375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:08.368834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:08.399384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9893a9ef-e495-4096-986c-95d761843c8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044034 May 19 12:30:08.399768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 815/1757] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:09.255879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:09.256492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:09.264101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:09.264539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:09.265310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:09.265733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:09.338634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:09.346332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f497943a-6b6a-4d86-972e-67ee8afe1c41 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096707 May 19 12:30:09.346921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 943/1758] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:09.415517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:09.417041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:09.428128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:09.428939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:09.430220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:09.430652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:09.460408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1d12228b-ec14-4dee-ada0-44b7a52cf6f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051513 May 19 12:30:09.461031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 816/1759] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:10.367611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:10.368339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:10.376193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:10.376760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:10.377543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:10.378008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:10.457757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:10.467678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d88196e9-f0b1-4bba-b130-f713b234a44f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.106753 May 19 12:30:10.468245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 944/1760] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:10.472286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:10.473240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:10.479671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:10.480113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:10.480923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:10.481350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:10.507238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:10.511951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-92039353-4eb8-495b-aa97-4072dee0e864 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044246 May 19 12:30:10.512474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 817/1761] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:11.483964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:11.484622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:11.491595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:11.492052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:11.492996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:11.493274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:11.523741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:11.524682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:11.532035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:11.532479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:11.533307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:11.533738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:11.560145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:11.564531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0699f203-2cfc-418f-81b5-f1f0c4f9979c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044734 May 19 12:30:11.565527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 818/1762] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:11.571833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-660e77df-5175-479f-a361-5aa636880e34 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091757 May 19 12:30:11.572436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 945/1763] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:12.580482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:12.581995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:12.592340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:12.593183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:12.593375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:12.593723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:12.594967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:12.595681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:12.604293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:12.604769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:12.605535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:12.605962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:12.629822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a8c3fdc0-828b-488c-91f7-75f12c4b2458 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055953 May 19 12:30:12.630390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 819/1764] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:12.677945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:12.685999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-257dc73a-fc7d-4089-bf18-74eafa4c19eb tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100142 May 19 12:30:12.686563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 946/1765] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:13.646336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:13.647817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:13.658704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:13.659441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:13.660718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:13.661430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:13.692912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-979ea3e4-f9ea-412d-8824-ca2a0f10e73e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053484 May 19 12:30:13.693468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 820/1766] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:13.703519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:13.704087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:13.710794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:13.711231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:13.711980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:13.712401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:13.716525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:13.775270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:13.784091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bd7fefc2-8c11-4b89-838d-43a87294e7cf tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084413 May 19 12:30:13.784657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 947/1767] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:30:14.709043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:14.710549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:14.719106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:14.719581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:14.720355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:14.720813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:14.750373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-508a2333-c8e4-4aa5-a41f-3d1f7481201d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047991 May 19 12:30:14.750922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 821/1768] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:14.803349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:14.804016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:14.813105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:14.813696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:14.814756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:14.815318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:14.882209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:14.890097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-039850d2-388a-4997-a2ee-7dad77f21919 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092072 May 19 12:30:14.890670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 948/1769] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:15.350129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:30:15.350432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:30:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:30:15.764020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:15.764902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:15.771729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:15.772173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:15.772948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:15.773370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:15.803047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-614bb2a5-3408-43d7-8776-c4a9eccf52c0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044090 May 19 12:30:15.803590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 822/1770] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:15.910967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:15.911949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:15.922943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:15.923681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:15.924994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:15.925708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:15.992955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:16.001309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-239847d5-9d15-4777-b3a3-09fb8a16a075 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096822 May 19 12:30:16.001888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 949/1771] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:16.818528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:16.819989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:16.831089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:16.831815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:16.833101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:16.833803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:16.871699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b603600e-1554-42a0-9054-d7de26e39c00 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059851 May 19 12:30:16.872289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 823/1772] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:17.018431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:17.019058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:17.025965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:17.026407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:17.027162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:17.027587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:17.089603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:17.101316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9381cdc5-a842-4959-90ff-7a57f9ab1ada tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086320 May 19 12:30:17.101988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 950/1773] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:17.888080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:17.889176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:17.897127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:17.897646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:17.898533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:17.899037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:17.931270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3f4eb00a-701c-4139-922a-13fe3dbb3af7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049930 May 19 12:30:17.931811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 824/1774] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:18.122890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:18.123929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:18.134971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:18.135703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:18.137028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:18.137741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:18.201162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:18.209083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-82a76dc0-8ff6-42ec-aa53-03a874ba7be0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093126 May 19 12:30:18.209651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 951/1775] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:18.949557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:18.951017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:18.961572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:18.962254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:18.963486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:18.964187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:18.993984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-357677df-d829-441f-848b-dbabe565355e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051028 May 19 12:30:18.994521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 825/1776] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:19.230264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:19.231207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:19.242078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:19.242810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:19.244043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:19.244740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:19.319070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:19.329581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-78bea31b-0915-4bd4-9cb2-14f877bdab15 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.105938 May 19 12:30:19.330159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 952/1777] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:20.010557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:20.012028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:20.019743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:20.020185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:20.020966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:20.021397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:20.051171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e65a8ae5-7cf8-40d0-bf6a-f86747f8a605 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047275 May 19 12:30:20.051740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 826/1778] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:20.350429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:20.351436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:20.360576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:20.361048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:20.361806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:20.362228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:20.426401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:20.434647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5de7c3e3-9826-4eb5-9f20-8555d25cb727 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090750 May 19 12:30:20.435256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 953/1779] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:21.062770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:21.063676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:21.065992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:21.071394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:21.071830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:21.072575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:21.073018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:21.103616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-097b1014-9678-46f0-84f4-414cc9e2ab27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044847 May 19 12:30:21.104171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 827/1780] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:21.457368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:21.458274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:21.468522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:21.468988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:21.469739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:21.470159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:21.535798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:21.543374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ecae08a7-08de-4a47-8fe8-0e0342c0ba4d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094291 May 19 12:30:21.543946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 954/1781] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:22.119677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:22.121178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:22.132515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:22.133311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:22.134675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:22.135377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:22.168330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4d7a7306-d739-44d2-adde-d5ca482a702b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055389 May 19 12:30:22.168904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 828/1782] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:22.560827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:22.561413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:22.570970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:22.571541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:22.572630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:22.573252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:22.643131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:22.654949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1314b631-09dd-43c6-8832-a21969e85be5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097827 May 19 12:30:22.655765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 955/1783] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:23.184496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:23.185992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:23.193438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:23.193902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:23.194697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:23.195151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:23.225173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-487d88d9-dd5a-4349-b66d-216dfb555f26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047225 May 19 12:30:23.225641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 829/1784] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:23.677342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:23.677931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:23.680280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=56 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:23.685883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:23.686323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:23.687065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:23.687483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:23.752943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:23.765246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92e40e0b-fb9a-4e02-9758-6cadd97fe860 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091185 May 19 12:30:23.766095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 956/1785] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:24.240707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:24.242215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:24.251915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:24.252371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:24.253148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:24.253571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:24.282293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ea65fbfb-d11a-4ccc-91e3-5c4b89891929 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048290 May 19 12:30:24.282843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 830/1786] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:24.791751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:24.792720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:24.800907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:24.801350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:24.802104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:24.802525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:24.866879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:24.879086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9f7906a6-9f28-43df-b62e-290bfc3988bc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094115 May 19 12:30:24.879945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 957/1787] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:25.297239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:25.298207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:25.305635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:25.306044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:25.306831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:25.307276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:25.338499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a1c1f100-24e2-4a79-939a-c069d5c7d115 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047247 May 19 12:30:25.339053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 831/1788] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:25.899342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:25.900206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:25.909088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:25.909562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:25.910322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:25.910749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:25.973608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:25.985465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e699d7c-b607-49e6-8235-35e96e925a3b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092262 May 19 12:30:25.986259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 958/1789] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:26.358834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:26.358834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:26.373007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:26.373478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:26.374298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:26.377198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:26.408420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-170f0a46-61af-42c5-8ce7-cfab19efccb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060436 May 19 12:30:26.409067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 832/1790] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:27.006996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:27.008209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:27.017629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:27.018104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:27.018854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:27.019278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:27.093755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:27.105487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cc699745-022d-45f9-9de9-1b391dc90a21 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.104562 May 19 12:30:27.106150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 959/1791] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:26 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:27.424012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:27.424865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:27.434907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:27.435400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:27.436421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:27.436883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:27.470276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a5f383fa-e8b8-4636-a38d-d54526b5fdac tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052432 May 19 12:30:27.470844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 833/1792] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:28.181051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:28.181974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:28.190521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:28.190959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:28.191699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:28.192118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:28.265080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:28.272819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8152eeb7-7e86-43de-a214-a7d0393a4d18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098166 May 19 12:30:28.273385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 960/1793] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:28.485305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:28.486133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:28.492935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:28.493375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:28.494132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:28.494552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:28.511886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:28.526227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9effc5c8-b035-4a95-94ef-4d0285acd64b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047206 May 19 12:30:28.526793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 834/1794] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:29.293161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:29.294130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:29.301596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:29.302034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:29.302777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:29.303262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:29.367124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:29.380288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1f002e88-1050-472b-9e3f-c4bde949e5a6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093202 May 19 12:30:29.381783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 961/1795] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:29.538360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:29.539272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:29.546275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:29.546652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:29.547409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:29.547836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:29.577876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4423b81a-462c-4275-b128-7cc5ef48c12f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043040 May 19 12:30:29.578354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 835/1796] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:30.402302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:30.402920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:30.409793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:30.410266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:30.411008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:30.411432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:30.472294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:30.480067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1963c3ac-09de-4583-8ea2-c2af220984c6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.081593 May 19 12:30:30.480663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 962/1797] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:30.590214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:30.591164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:30.598995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:30.599356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:30.600345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:30.600875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:30.631375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f900f5e7-0696-4cb4-89ec-e632c5317457 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045167 May 19 12:30:30.632065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 836/1798] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:31.510139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:31.511778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:31.523237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:31.524002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:31.525394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:31.526150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:31.596446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:31.601945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:31.608931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d2f43e80-e6de-49fc-9eed-f00c0dfe35b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.112999 May 19 12:30:31.609510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 963/1799] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:31.644265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:31.645242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:31.652178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:31.652626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:31.653413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:31.653831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:31.683710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cc03a3c-1e59-4617-be48-b7f36cd0b1a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043444 May 19 12:30:31.684264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 837/1800] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:32.627282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:32.627997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:32.636882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:32.637405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:32.638298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:32.638841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:32.695748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:32.696714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:32.703966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:32.703966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:32.704688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:32.705144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:32.706295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:32.714153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fcda88bd-03c8-4f47-a402-7afb45ba734b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092095 May 19 12:30:32.714973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 964/1801] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:32.735369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0f2f6850-e918-4583-93b9-f85cd762e1f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043631 May 19 12:30:32.735937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 838/1802] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:33.734820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:33.735493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:33.744207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:33.744775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:33.745672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:33.746177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:33.748679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:33.749779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:33.757708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:33.758150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:33.759071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:33.759567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:33.791355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-179562d8-cddd-4b14-8c89-dd13eb3b3f58 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047661 May 19 12:30:33.791935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 839/1803] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:33.814296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:33.822361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4caf0138-4461-42be-ae23-83f411befb8f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093616 May 19 12:30:33.822935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 965/1804] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:34.805880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:34.806968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:34.815096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:34.815537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:34.816303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:34.816770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:34.838913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:34.839509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:34.846661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:34.847104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:34.847263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ad10a0f5-66d0-4d1a-a94b-2143386f0fa6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046980 May 19 12:30:34.847871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 840/1805] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:34.848034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:34.848299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:34.910259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:34.918177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1f46897c-c616-40ca-81bd-04416adace90 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083169 May 19 12:30:34.918733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 966/1806] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:34 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:35.861453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:35.862374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:35.869281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:35.869725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:35.870479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:35.870913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:35.900552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f40b5bb5-f756-4198-a7b3-ba7cddbe8776 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043931 May 19 12:30:35.901126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 841/1807] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:35.937594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:35.938282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:35.947461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:35.948077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:35.949153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:35.949844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:36.012683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:36.021004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e754e636-aa1b-4e04-96cc-d0cace546d80 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089565 May 19 12:30:36.021608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 967/1808] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:36.916519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:36.917509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:36.925837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:36.926281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:36.927047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:36.927581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:36.958877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-095d66a2-f355-4cd4-98eb-ac5a41b6cb03 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048009 May 19 12:30:36.959642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 842/1809] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:37.037855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:37.038463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:37.045280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:37.045722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:37.046475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:37.046907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:37.110028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:37.117750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3a019ad-be8b-49c2-9ecb-7980a051e4be tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083740 May 19 12:30:37.118297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 968/1810] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:37.971094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:37.972579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:37.979350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:37.979800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:37.980559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:37.981018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:38.012016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3550394c-f259-4cbc-8a1a-b3023f3bfa88 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045160 May 19 12:30:38.012579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 843/1811] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:37 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:38.135459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:38.135981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:38.142707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:38.143152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:38.143896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:38.144346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:38.209427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:38.217524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9b57e137-38c8-4617-905f-b75a3a5ccd08 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086058 May 19 12:30:38.218099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 969/1812] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:39.028199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:39.029882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:39.040911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:39.041645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:39.042936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:39.043636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:39.074045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e4bad57-6229-455f-bc97-e9966aac1721 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052401 May 19 12:30:39.074609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 844/1813] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:39.238736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:39.239739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:39.250620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:39.251043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:39.251822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:39.252252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:39.322969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:39.330834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bb6646c6-7863-4173-8c13-b7055c137ad7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098823 May 19 12:30:39.331408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 970/1814] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:40.086528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:40.087506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:40.094008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:40.094526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:40.095348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:40.095840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:40.126189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7c6de0a1-bc91-4e41-a55f-cde411818201 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044111 May 19 12:30:40.126751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 845/1815] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:40.347839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:40.348570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:40.355508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:40.355969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:40.356741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:40.357164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:40.419214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:40.427207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-82dfabf8-8e31-4985-b080-a46d01d37085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083302 May 19 12:30:40.427783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 971/1816] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:41.141119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:41.141954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:41.149141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:41.149580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:41.150332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:41.150757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:41.180056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e34293cd-a8f1-4a68-94a6-bde8b03b679e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044676 May 19 12:30:41.180601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 846/1817] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:41.447530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:41.448394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:41.459897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:41.460679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:41.461940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:41.462713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:41.534513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:41.542305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-93f01957-683c-4c4c-97ad-03706d2c12b6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101374 May 19 12:30:41.542865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 972/1818] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:42.196141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:42.197563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:42.208698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:42.209445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:42.210683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:42.211525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:42.244252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-92284772-5da4-4720-b83c-10f5be0104e2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054752 May 19 12:30:42.244848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 847/1819] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:42.561845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:42.562487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:42.572854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:42.573476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:42.574629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:42.575183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:42.637385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:42.645412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d20eb74-263a-4647-90e6-c95a023e76f5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089656 May 19 12:30:42.645975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 973/1820] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:43.255779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:43.256741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:43.263397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:43.263832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:43.264673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:43.265103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:43.295083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58a59fba-7154-47d4-b1aa-9ebde80e4bd4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042535 May 19 12:30:43.295411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 848/1821] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:43.666820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:43.667836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:43.679058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:43.679835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:43.681089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:43.681876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:43.752394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:43.760433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b490631-eb0f-4f02-8b8f-d6835ade2c6f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100154 May 19 12:30:43.761039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 974/1822] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:44.306613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:44.307557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:44.314431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:44.314876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:44.315629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:44.316054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:44.350018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d3f6c0c9-ea3e-4da6-9d08-c4c948fed2ab tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047110 May 19 12:30:44.350595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 849/1823] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:44.782616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:44.783388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:44.790449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:44.790963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:44.791693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:44.792125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:44.858433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:44.866356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4ba820ab-6b8d-49f0-81f4-e8a9f58e840d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087604 May 19 12:30:44.866936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 975/1824] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:45.366350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:45.367862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:45.377249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:45.377688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:45.378445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:45.378866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:45.408142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c28b86bf-657b-46a8-badf-10e5562c4331 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048551 May 19 12:30:45.408801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 850/1825] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:45.887408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:45.888386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:45.899686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:45.900431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:45.901732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:45.902448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:45.969010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:45.980497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-227ed38e-49c1-43e4-86b3-745be33d62ff tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099357 May 19 12:30:45.981307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 976/1826] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:45 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:46.423653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:46.424813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:46.433139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:46.433689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:46.434617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:46.435168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:46.439790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:46.468788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-62444a32-b6a6-45f2-ad37-04230c63bb27 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050953 May 19 12:30:46.469358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 851/1827] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:47.001582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:47.002202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:47.008980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:47.009424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:47.010182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:47.010604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:47.074643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:47.083163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1d29b038-ab18-42bc-811f-be0caad462b4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087579 May 19 12:30:47.083725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 977/1828] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:46 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:47.484205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:47.485705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:47.496697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:47.497778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:47.498966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:47.499457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:47.529091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c2712fa-295d-4d16-a788-ae27939bf25d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051589 May 19 12:30:47.529645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 852/1829] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:48.103510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:48.104516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:48.112798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:48.113245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:48.114002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:48.114434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:48.178948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:48.186731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ba3c9eb0-a9cb-45ab-b6ce-9596fea4fcc6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089823 May 19 12:30:48.187289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 978/1830] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:48.541022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:48.541945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:48.548745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:48.549190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:48.549951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:48.550379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:48.580471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07bc41ba-09ba-4d92-b16c-7cbb555ef4f3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043457 May 19 12:30:48.581065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 853/1831] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:49.205637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:49.206376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:49.215324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:49.215910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:49.216951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:49.217572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:49.297077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:49.304956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8580e48e-aa2d-4858-9c43-b37c28a44ef2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.104515 May 19 12:30:49.305532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 979/1832] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:49.596717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:49.598381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:49.608961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:49.609529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:49.610163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:49.610574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:49.641541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-95b61aef-3d6b-486c-8012-dd45fc616752 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051578 May 19 12:30:49.642097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 854/1833] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:50.321657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:50.322275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:50.329421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:50.329860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:50.330617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:50.331043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:50.349637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:50.622229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:50.630255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28e44553-42c1-4a51-b54e-1c333fe36bb3 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.312584 May 19 12:30:50.630823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 980/1834] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:50.653126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:50.654056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:50.660971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:50.661423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:50.662181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:50.662611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:50.692959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-63a55d1f-e017-4d25-941b-faf7c5765a09 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043690 May 19 12:30:50.693519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 855/1835] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:51.647707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:51.648299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:51.655432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:51.655866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:51.656667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:51.657093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:51.704573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:51.705544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:51.708093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:51.713844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:51.714388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:51.715171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:51.715632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:51.724404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:51.732573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bc988105-fa26-4613-9106-d4597af3a76b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088794 May 19 12:30:51.733252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 981/1836] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:51.748557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-50fdff2c-8261-4999-b150-23fff5efff9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048030 May 19 12:30:51.749133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 856/1837] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:52.754416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:52.755384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:52.764423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:52.766157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:52.766403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:52.767222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:52.768681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:52.769251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:52.777006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:52.777713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:52.779014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:52.779733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:52.816338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34667646-12b9-4d55-8b88-4473b0b9865d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058538 May 19 12:30:52.817114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 857/1838] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:52.842365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:52.850350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c8282b1f-969a-4416-a4a9-9beaaba7e263 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102609 May 19 12:30:52.850902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 982/1839] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:53.827813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:53.828755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:53.836039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:53.836491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:53.837549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:53.837977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:53.867692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:53.868420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:53.868993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fb39cb24-c275-4d32-b929-c88814177ba8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045187 May 19 12:30:53.869556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 858/1840] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:53.870800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:53.876473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:53.876962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:53.877712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:53.878138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:53.940180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:53.948133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6859227a-170f-48ae-9ad7-89c3cf7e9f0a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084455 May 19 12:30:53.948719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 983/1841] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:54.881183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:54.882205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:54.889886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:54.890328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:54.891098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:54.891543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:54.921904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-75dc5fab-d99b-4a50-b20d-c9b3309d9516 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045016 May 19 12:30:54.922494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 859/1842] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:54.966891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:54.967458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:54.974648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:54.975187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:54.975941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:54.976358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:55.041566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:55.051280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dcef038c-4f77-4be1-bed0-1d0db7d4ed4c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088254 May 19 12:30:55.051837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 984/1843] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:30:55.934220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:55.935200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:55.942405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:55.942852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:55.943631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:55.944083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:55.977221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-77a8af31-f51e-440b-b82a-6bc92aa829b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046932 May 19 12:30:55.977784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 860/1844] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:56.067692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:56.068258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:56.075282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:56.075737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:56.076487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:56.076963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:56.143707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:56.152478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ce072e40-27e2-414e-8e50-4acac48b475f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088742 May 19 12:30:56.153374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 985/1845] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:56.990137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:56.991042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:56.998869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:56.999397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:57.000420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:57.001046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:57.038539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-705b0454-4307-48d9-a633-2eb905def97c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052649 May 19 12:30:57.039105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 861/1846] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:57.169764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:57.170361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:57.177590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:57.178025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:57.178776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:57.179191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:57.242930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:57.251251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bda98997-c50b-4d1d-ae48-c2760b921823 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085469 May 19 12:30:57.252055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 986/1847] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:58.058339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:58.059327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:58.067199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:58.067686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:58.068497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:58.069082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:58.105793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7ac2f50f-3897-4723-9f2d-6facfd5ab0c5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057021 May 19 12:30:58.106366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 862/1848] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:58.268011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:58.268671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:58.275804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:58.276252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:58.277124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:58.277560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:58.341433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:58.349456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e2a1da2e-2049-4f1d-a71d-153bdb0a2c8b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085567 May 19 12:30:58.350064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 987/1849] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:30:59.122527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:59.123436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:59.130424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:59.130868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:59.131664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:59.132121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:59.164517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ef8c56d1-4e47-45c7-aa47-f7f939f2feb5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046235 May 19 12:30:59.165152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 863/1850] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:30:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:30:59.366697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:30:59.367309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:30:59.374724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:59.375253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:59.375596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:30:59.376261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:30:59.441379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:30:59.450127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0fd92c4b-506e-41e2-939a-3187955ec18d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087430 May 19 12:30:59.450880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 988/1851] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:30:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:00.180155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:00.181435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:00.188560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:00.189144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:00.189983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:00.190633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:00.232592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3ce9bff6-4e01-43a0-a273-b250f3662731 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060189 May 19 12:31:00.233633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 864/1852] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:00.467761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:00.468606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:00.474801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:00.475295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:00.476064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:00.476459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:00.545552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:00.553111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dbe578c7-db8c-46b8-baef-71e79ecf8dde tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.089772 May 19 12:31:00.553679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 989/1853] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:01.249610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:01.251078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:01.261952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:01.262737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:01.263976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:01.264748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:01.298940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-63ab4210-b561-4546-ba73-504ee3635d5b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056541 May 19 12:31:01.299485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 865/1854] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:01.574117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:01.575076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:01.582811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:01.583251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:01.583988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:01.584405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:01.645597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:01.653388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e86ac62-006b-4021-a1ff-cce9e932ab47 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085814 May 19 12:31:01.653928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 990/1855] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:02.315024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:02.316493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:02.327573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:02.328015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:02.328814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:02.329275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:02.358494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e8d795a7-60f2-42de-9b5f-bda64c2ec304 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050252 May 19 12:31:02.359167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 866/1856] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:02.674029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:02.674879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:02.685778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:02.686508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:02.687459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:02.687873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:02.751263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:02.762640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2c5ebc48-30b5-4c0e-a923-4398cf1f6faa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094999 May 19 12:31:02.763617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 991/1857] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:03.375133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:03.376056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:03.383163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:03.383609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:03.384372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:03.384829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:03.409394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:03.414041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afc7fbef-883d-4e13-b803-0a03c0f61b83 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045673 May 19 12:31:03.414626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 867/1858] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:03.783015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:03.783846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:03.793840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:03.794520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:03.795623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:03.796295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:03.861354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:03.868933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-00dc9ebe-5b9b-4820-82b7-dc7eec8a25d8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091478 May 19 12:31:03.869506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 992/1859] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:04.430884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:04.432353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:04.443239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:04.444009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:04.445287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:04.446000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:04.478157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eafd1d0b-ebd7-4e16-8a78-55354ba01d4b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054363 May 19 12:31:04.478718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 868/1860] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:04.890073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:04.890998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:04.901991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:04.902714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:04.903992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:04.904705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:04.965846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:04.973927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-31faaee9-0226-474c-8d3c-944df7caef5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090374 May 19 12:31:04.974499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 993/1861] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:05.490305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:05.491243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:05.498347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:05.498886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:05.499742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:05.500286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:05.529785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a8d7ba0-42c0-41ba-bb6a-0aa223b61ed3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043457 May 19 12:31:05.530342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 869/1862] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:05.998856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:05.999806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:06.010752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:06.012145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:06.013498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:06.014221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:06.076323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:06.083939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-494901de-47cf-4e17-9213-8924f1e88639 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092032 May 19 12:31:06.084506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 994/1863] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:05 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:06.543821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:06.544775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:06.551803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:06.552365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:06.553161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:06.553589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:06.588257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1ddd74b7-919e-407c-9b10-a96ff6392264 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048501 May 19 12:31:06.588849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 870/1864] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:07.104387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:07.105512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:07.116419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:07.117196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:07.118482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:07.119183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:07.186894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:07.194819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ada9ff30-58d8-45ab-9210-a85701d4dd26 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097242 May 19 12:31:07.195384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 995/1865] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:07.603802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:07.605330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:07.613523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:07.613969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:07.614726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:07.615171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:07.651407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7f72d9bd-f5f6-4e09-87b7-1ac26cf304b9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053943 May 19 12:31:07.652151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 871/1866] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:08.216047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:08.217123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:08.227331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:08.228062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:08.229352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:08.230237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:08.298885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:08.306771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e915920d-34bd-41ea-b7cc-1dcb48361402 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097344 May 19 12:31:08.307347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 996/1867] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:08.667768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:08.669285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:08.680332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:08.681229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:08.682548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:08.683259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:08.714174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fccae972-5152-49b3-92bc-a82a2a832b8d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053107 May 19 12:31:08.714731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 872/1868] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:09.328416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:09.329459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:09.340260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:09.341056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:09.342301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:09.343153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:09.347200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:09.405229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:09.413446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-94397768-e9e4-4d5c-944b-18012ccb90ed tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091399 May 19 12:31:09.414023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 997/1869] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:09.730370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:09.731901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:09.739777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:09.740231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:09.741053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:09.741487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:09.771235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c7cc1ec7-107b-486f-93fe-a61bdf252500 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047634 May 19 12:31:09.771721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 873/1870] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:10.437577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:10.438568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:10.449705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:10.450446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:10.451870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:10.452677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:10.516084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:10.527247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b59cbda8-b776-478f-aeeb-f04001613f75 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095872 May 19 12:31:10.527973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 998/1871] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:10.788001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:10.789409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:10.800313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:10.801111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:10.802363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:10.803084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:10.835682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b931c37-18cd-458f-8dea-9bbea730bfbb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054480 May 19 12:31:10.836232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 874/1872] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:11.553135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:11.553724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:11.562743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:11.563182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:11.563924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:11.564347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:11.637546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:11.645110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d482b912-bc74-4b7f-928a-7906ced0905d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098197 May 19 12:31:11.645671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 999/1873] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:11.851171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:11.852685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:11.862989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:11.863444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:11.864208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:11.864669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:11.895166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-856d9734-0b7f-4118-bb5c-350b3376541a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050591 May 19 12:31:11.895742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 875/1874] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:12.665613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:12.666601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:12.675518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:12.675973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:12.676765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:12.677193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:12.749132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:12.757711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-25863b32-bf8c-4567-a226-8cf7fd0f72e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098561 May 19 12:31:12.758288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1000/1875] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:12.913021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:12.914496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:12.926824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:12.927565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:12.928878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:12.929591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:12.962054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d4c65560-28bc-41f4-9fa4-dab6b54159d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055765 May 19 12:31:12.962607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 876/1876] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:13.778877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:13.779772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:13.787361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:13.787806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:13.788554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:13.789010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:13.857910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:13.868211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db59fbae-1d2b-406e-943c-d7ba1e3d171b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095983 May 19 12:31:13.868812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1001/1877] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:13.974929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:13.975832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:13.983085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:13.983528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:13.984326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:13.984818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:14.014393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5ba81f3d-28b4-46d5-b2ae-904419da689b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043490 May 19 12:31:14.014943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 877/1878] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:14.889532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:14.890437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:14.898029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:14.898519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:14.899353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:14.899826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:14.962920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:14.970663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a9de6a8a-61dc-4173-89c8-af67a57e9459 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087592 May 19 12:31:14.971236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1002/1879] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:15.030417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:15.031874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:15.042712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:15.043449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:15.044715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:15.045437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:15.074871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b501b0b6-9906-43cd-aa5c-44ae1e26f287 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051207 May 19 12:31:15.075418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 878/1880] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:15.987895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:15.988462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:15.995628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:15.996067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:15.996859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:15.997634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:16.061813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:16.070119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-88d31c36-6547-4925-8acf-2ace96f630dc tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086280 May 19 12:31:16.070691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1003/1881] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:16.087819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:16.089124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:16.098102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:16.098559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:16.099384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:16.099923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:16.129593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-60078619-59f8-4261-afa8-5f1946e08fdd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046895 May 19 12:31:16.130153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 879/1882] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:17.094808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:17.095870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:17.106837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:17.107575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:17.108881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:17.109709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:17.143158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:17.144220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:17.151332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:17.151692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:17.152558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:17.153020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:17.177027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:17.182946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dc3a99bc-d9b0-4ec3-a40a-34dd1e41049b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043770 May 19 12:31:17.183520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 880/1883] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:17.185394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9249f14f-7ac0-4abd-9755-651546c5f92c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097256 May 19 12:31:17.185954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1004/1884] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:18.199287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:18.200852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:18.206172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:18.207168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:18.211638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:18.211911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:18.213042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:18.213752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:18.217244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:18.217954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:18.219211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:18.219632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:18.246137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ed3749ce-c829-461e-a629-db50bc59778d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053935 May 19 12:31:18.246706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 881/1885] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:18.292316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:18.300540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-12833ffc-cfaf-4ecc-ac38-2db237abca4f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100812 May 19 12:31:18.301108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1005/1886] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:19.262614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:19.264091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:19.271797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:19.272235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:19.273108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:19.273559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:19.302706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8d6f2f88-58b5-4c7a-8c2f-f8efc281e84f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046916 May 19 12:31:19.303256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 882/1887] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:19.323897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:19.324484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:19.332442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:19.332922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:19.333664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:19.334085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:19.402022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:19.409905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2738351f-61a0-4a58-93e6-bb9e0c4b919b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091682 May 19 12:31:19.410456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1006/1888] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:20.318731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:20.320225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:20.330257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:20.330703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:20.331468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:20.331894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:20.361385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ecfa22f-cb68-40a4-81cd-ac57b43a1920 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049381 May 19 12:31:20.361954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 883/1889] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:20.430721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:20.431726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:20.440989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:20.441430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:20.442186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:20.442612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:20.512668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:20.520973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e04bc9a-b393-43bf-b498-0046a040fa11 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096617 May 19 12:31:20.521536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1007/1890] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:21.378141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:21.379121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:21.386436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:21.386878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:21.387654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:21.388080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:21.405151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:21.419439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34e25dc5-be1f-44d4-8c9a-7b79b7e28b86 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047390 May 19 12:31:21.420030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 884/1891] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:21.542212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:21.542821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:21.552180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:21.552625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:21.553423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:21.553848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:21.620773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:21.632578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d6f5c3f8-1501-43fd-a171-5429b3c33fc7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096663 May 19 12:31:21.633499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1008/1892] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:22.433158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:22.434213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:22.436903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:22.443810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:22.444307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:22.445212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:22.445698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:22.477094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa938cd3-0035-4cb3-8fc1-7458f8279ad9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048470 May 19 12:31:22.477568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 885/1893] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:22.656669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:22.657302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:22.665781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:22.666228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:22.666982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:22.667409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:22.729807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:22.737997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b0a9f845-4cdf-40f6-a7b8-5fe81e424455 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087423 May 19 12:31:22.738561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1009/1894] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:23.494255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:23.495846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:23.506554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:23.507088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:23.507982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:23.508478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:23.541971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-df2a772c-ab56-4ff8-bbaa-d01c046c7b6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055247 May 19 12:31:23.542506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 886/1895] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:31:23.761057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:23.761635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:23.768667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:23.769123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:23.769884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:23.770314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:23.833025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:23.841021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ab126040-57b5-41b4-81cd-bf7cb7b286e6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084013 May 19 12:31:23.841593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1010/1896] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:24.555083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:24.556074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:24.565088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:24.565541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:24.566319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:24.566751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:24.606003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5ab0f5d1-d17f-432b-ac05-c5f4b294df0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055120 May 19 12:31:24.606689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 887/1897] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:24.863039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:24.863822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:24.867986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=56 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:24.877547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:24.878218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:24.879371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:24.880015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:24.946307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:24.954293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65bc072e-4deb-461b-b83e-2b3544f756c9 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098584 May 19 12:31:24.954895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1011/1898] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:25.621220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:25.623283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:25.632187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:25.632909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:25.634057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:25.634659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:25.665283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba66eb64-e604-41c5-b147-9f78e82dba9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048082 May 19 12:31:25.665879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 888/1899] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:25.976034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:25.977060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:25.995078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:25.995530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:25.996302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:25.996754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:26.076717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:26.084845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d00bbe75-cc00-410a-8dfc-120db3badb52 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.115036 May 19 12:31:26.085492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1012/1900] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:26.682935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:26.683860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:26.690836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:26.691910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:26.692668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:26.693129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:26.724698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6630a3dc-338f-4557-8da8-1c543114f391 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047979 May 19 12:31:26.725462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 889/1901] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:27.262490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:27.263275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:27.272310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:27.273173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:27.274398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:27.275086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:27.342384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:27.347400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:27.354181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-575591e7-44c1-4f05-afdc-f1d9f120d26a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.254268 May 19 12:31:27.355001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1013/1902] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:27.736567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:27.737509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:27.744367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:27.744883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:27.745625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:27.746052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:27.775757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aae1eefc-e5f3-4fae-ab5f-c488c4863001 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043135 May 19 12:31:27.776226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 890/1903] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:28.380253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:28.381243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:28.389632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:28.390070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:28.390814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:28.391266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:28.452932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:28.460923 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-321468e1-ff18-4739-9b7a-0701f2e52ed0 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087644 May 19 12:31:28.461479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1014/1904] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:28.791930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:28.793354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:28.804433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:28.805218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:28.806456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:28.807219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:28.837747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c4ca3a2f-1286-4233-b72d-54ea89cab59e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052637 May 19 12:31:28.838325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 891/1905] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:29.483905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:29.484746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:29.493827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:29.494405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:29.495399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:29.496065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:29.562673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:29.570350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d27a694d-b977-4a5d-996b-348d28ab7bc1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093819 May 19 12:31:29.570927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1015/1906] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:29.853418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:29.854891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:29.862345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:29.862825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:29.863615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:29.864065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:29.894662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-efb5c230-601b-4e3a-9aee-8885c74802f6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047982 May 19 12:31:29.895215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 892/1907] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:30.591312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:30.592499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:30.603514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:30.604231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:30.605568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:30.605993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:30.675259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:30.683510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ddaee63f-6234-4e83-a8fd-d73679bc51cd tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098722 May 19 12:31:30.684070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1016/1908] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:30 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:30.910787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:30.912303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:30.923288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:30.924025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:30.925328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:30.926042 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:30.958604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8b5c7d13-d85e-4e8d-bee9-ae6b66525e25 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054412 May 19 12:31:30.959152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 893/1909] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:31.710828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:31.712213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:31.722667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:31.723323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:31.724877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:31.725388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:31.791178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:31.801049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4fee308e-e4cc-478d-98be-5c8d20d368c4 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102749 May 19 12:31:31.802116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1017/1910] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:31 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:31.974451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:31.975928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:31.984472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:31.984952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:31.985704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:31.986128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:32.015011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c0f733f2-59d3-430a-8536-3b68fd72f0b4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047145 May 19 12:31:32.015545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 894/1911] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:32.822705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:32.823665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:32.830987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:32.831425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:32.832179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:32.832602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:32.898851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:32.908017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-74cbea91-d691-445c-838d-69d22d1b614d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091921 May 19 12:31:32.908589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1018/1912] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:32 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:33.029620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:33.030541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:33.037117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:33.037560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:33.038316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:33.038741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:33.067242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-55ceb9c6-7673-44af-b91b-f8b95aadecc7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043142 May 19 12:31:33.067784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 895/1913] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:33.928951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:33.929954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:33.938196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:33.938637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:33.939394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:33.939816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:34.006587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:34.014766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ceec26e3-9808-47ae-952f-79b3a35936ab tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092463 May 19 12:31:34.015351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1019/1914] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:33 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:34.090235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:34.091919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:34.102841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:34.103587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:34.104898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:34.105709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:34.137787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cdcb9ce2-ce8f-444a-af4f-56245d7eb99b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057556 May 19 12:31:34.138354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 896/1915] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:35.035727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:35.036707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:35.046002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:35.046467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:35.047217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:35.047640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:35.111657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:35.119981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b9ff3bb9-465a-4d3c-b337-40c2b1061032 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090886 May 19 12:31:35.120533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1020/1916] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:35 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:35.149724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:35.150631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:35.158022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:35.158460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:35.159203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:35.159626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:35.189525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ce622c85-6338-4f1d-8a3b-dd31b4ed2cce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043681 May 19 12:31:35.190075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 897/1917] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:36.138842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:36.139511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:36.147610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:36.148137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:36.149047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:36.149573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:36.203684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:36.204993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:36.215880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:36.216335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:36.217159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:36.217573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:36.222268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:36.231037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5dcf79a-0504-4690-b005-5a47b41ca8b8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096876 May 19 12:31:36.232138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1021/1918] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:36 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:36.254326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e03375b5-ca84-4d06-9e22-48499b253a21 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056795 May 19 12:31:36.254781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 898/1919] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:37.248675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:37.249271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:37.257500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:37.257990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:37.258856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:37.259345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:37.266489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:37.267414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:37.273901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:37.274361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:37.275102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:37.275516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:37.305722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-629af84f-d184-45bd-bab5-3da3c45f248b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043174 May 19 12:31:37.306305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 899/1920] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:37 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:37.320797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:37.328763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0ca53028-e819-483f-9b3b-a90480a320ba tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085017 May 19 12:31:37.329324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1022/1921] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:37 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:38.318579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:38.319638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:38.327910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:38.328460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:38.329420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:38.330138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:38.345995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:38.346606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:38.353351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:38.353802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:38.354565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:38.354995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:38.361317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-117c4a32-a2ae-408b-a574-c1bb113b95d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047379 May 19 12:31:38.361885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 900/1922] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:38.418569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:38.426758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a9da9680-23ad-4bfd-95c3-7f6750fbde18 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.084710 May 19 12:31:38.427353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1023/1923] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:38 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:39.377041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:39.378559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:39.389709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:39.390446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:39.391414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:39.391847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:39.395513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:39.421563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a7237eb-0754-49db-a86a-ea8e866989f1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051531 May 19 12:31:39.422109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 901/1924] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:39.443425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:39.444128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:39.451756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:39.452187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:39.452968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:39.453390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:39.518002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:39.525659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-68bf77fe-2c8f-4fa5-94f2-87b411945f61 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086137 May 19 12:31:39.526214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1024/1925] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:39 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:40.437014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:40.438468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:40.449786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:40.450519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:40.451758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:40.452459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:40.487180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3a802e10-976f-4cc4-9516-e329e2165afe tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057019 May 19 12:31:40.487747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 902/1926] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:40.545749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:40.546516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:40.557393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:40.557987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:40.559011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:40.559445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:40.634019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:40.641683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90aa1f31-d71d-4af1-b4dc-8635ed2063b1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.102788 May 19 12:31:40.642245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1025/1927] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:40 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:41.504706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:41.506403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:41.515777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:41.516287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:41.517173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:41.517651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:41.546808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a95c4f05-d00b-43da-b954-ddabf9ca0e6e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050081 May 19 12:31:41.547383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 903/1928] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:41.662410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:41.663083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:41.671959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:41.672407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:41.673275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:41.673704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:41.740696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:41.748887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ca88082f-090c-493d-952c-8c9c90f35826 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092884 May 19 12:31:41.749456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1026/1929] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:41 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:42.563377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:42.564904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:42.575299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:42.575743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:42.576497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:42.576958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:42.607741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cea5a164-cb59-4430-8b46-966279fc3d54 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051462 May 19 12:31:42.608296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 904/1930] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:42.770059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:42.770913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:42.780135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:42.780791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:42.781510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:42.781928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:42.852069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:42.859789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d999d2c2-2fc5-4577-93f4-e6c0abbfcbe2 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096413 May 19 12:31:42.862827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1027/1931] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:42 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:43.619574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:43.620518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:43.627865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:43.628313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:43.629103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:43.629531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:43.659392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-16a1ba57-496c-4c00-8a74-c6d7315b031c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043836 May 19 12:31:43.659946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 905/1932] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:43.882241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:43.883068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:43.891881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:43.892316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:43.893098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:43.893520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:43.965125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:43.973378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cf5dd6d6-56c2-4f56-98b2-a4bdec534ad6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097634 May 19 12:31:43.973942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1028/1933] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:43 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:44.677318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:44.678785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:44.689922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:44.690666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:44.691915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:44.692633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:44.725771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-08e0c562-a2be-42d0-ab39-1f0a3b745341 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055176 May 19 12:31:44.726337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 906/1934] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:44.994515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:44.995554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:45.003939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:45.004451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:45.005339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:45.005839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:45.071798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:45.079957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f0f4de52-4cf8-45c5-b512-3c39286bb966 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092106 May 19 12:31:45.080957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1029/1935] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:44 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:45.739093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:45.740393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:45.749129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:45.749581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:45.750350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:45.750785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:45.780246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ca176d5f-72ac-4dfe-aa8d-8874bb46781d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046115 May 19 12:31:45.780841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 907/1936] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:46.100911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:46.101871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:46.110798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:46.111239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:46.112049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:46.112475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:46.131142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:46.178107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:46.186356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b588804f-4b04-4c68-88bf-19130521b474 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.091865 May 19 12:31:46.186961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1030/1937] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:46 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:46.795916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:46.796903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:46.804801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:46.805320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:46.806201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:46.806701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:46.838895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8bd69ad2-2ff4-43f9-8e89-cb94c6340ec6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049854 May 19 12:31:46.839555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 908/1938] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:47.202920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:47.203535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:47.210432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:47.210876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:47.211728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:47.212053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:47.275860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:47.287242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-90d56392-0aac-4851-a265-7b6c3caa3129 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088222 May 19 12:31:47.287829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1031/1939] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:47 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:47.855402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:47.857003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:47.868092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:47.868857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:47.870118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:47.870846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:47.901728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ca1fb9dd-a983-4bfc-8a2f-35107bdc79cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053023 May 19 12:31:47.902259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 909/1940] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:48.308147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:48.308746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:48.318612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:48.319060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:48.319947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:48.320475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:48.382664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:48.390375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e1d5c38-9a64-4d3d-a5c2-1edefc81ac9e tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088455 May 19 12:31:48.390956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1032/1941] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:48 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:48.919815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:48.921318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:48.931795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:48.932530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:48.933823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:48.934544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:48.964845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-71ce92fa-fed2-450d-9582-478f70415d55 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051914 May 19 12:31:48.965395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 910/1942] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:49.411411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:49.412359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:49.423349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:49.424091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:49.425387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:49.426094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:49.492281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:49.500758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-84272d88-7103-4d94-a446-43ddad650309 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095858 May 19 12:31:49.501536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1033/1943] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:49 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:49.979829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:49.981332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:49.988573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:49.988979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:49.989736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:49.990164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:50.022081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b7807e96-af8c-483c-91eb-84fb9dbcdabb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048961 May 19 12:31:50.022648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 911/1944] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:50.518872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:50.519487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:50.526443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:50.526883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:50.527631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:50.528055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:50.596008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:50.607253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5f2b1e9b-2b0f-487f-b0c2-2bf9c7b691d1 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092103 May 19 12:31:50.607792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1034/1945] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:50 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:51.037209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:51.038397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:51.048860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:51.049485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:51.050833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:51.051256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:51.084005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7e1d6293-20c4-4ceb-8d17-9380357dff84 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052796 May 19 12:31:51.084555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 912/1946] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:51.628293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:51.629347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:51.640196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:51.641019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:51.642244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:51.642951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:51.709039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:51.717085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-08571aa7-f1fb-4e38-84d8-56862729ca20 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095554 May 19 12:31:51.717621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1035/1947] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:51 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:52.099871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:52.101319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:52.110135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:52.110580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:52.111340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:52.111766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:52.141325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-690a8e15-d8ac-4804-8998-2bdcea435f47 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048192 May 19 12:31:52.141914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 913/1948] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:52.735989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:52.736691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:52.744731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:52.745245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:52.746111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:52.746605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:52.811169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:52.818897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3942f502-272d-41db-8766-99223a5b3ed6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088224 May 19 12:31:52.819451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1036/1949] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:52 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:53.155967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:53.156840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:53.159283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:53.164698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:53.165184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:53.165933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:53.166366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:53.196479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b68bdc42-8a0c-43e0-b86c-6efd8a28199e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046657 May 19 12:31:53.197077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 914/1950] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:53.839845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:53.840832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:53.848330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:53.848822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:53.849565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:53.849984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:53.912460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:53.920063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-13389fbc-e306-444e-be16-69fde2ed1db7 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086732 May 19 12:31:53.920695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1037/1951] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:53 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:54.212683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:54.214045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:54.225048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:54.225788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:54.227045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:54.227756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:54.260378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e0c2a750-ee5f-4ec5-89c0-035b7158c000 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054494 May 19 12:31:54.260957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 915/1952] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:54.940998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:54.941968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:54.945607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:54.953543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:54.953987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:54.954737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:54.955164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:55.030523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:55.038828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ecf6b55-f441-4dc8-837e-bfcdcee2343f tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.104369 May 19 12:31:55.039413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1038/1953] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:54 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:55.275833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:55.277199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:55.288191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:55.288994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:55.290229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:55.291022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:55.321972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7117cd61-93f7-4620-94f5-14dc1a2edaf6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052943 May 19 12:31:55.322546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 916/1954] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:56.059966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:56.061050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:56.068016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:56.068457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:56.069245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:56.069671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:56.133009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:56.140926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f3c370f0-cf8c-47a4-9349-f3a5eae9f0f6 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087333 May 19 12:31:56.141492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1039/1955] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:56 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:56.333813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:56.334636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:56.341938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:56.342515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:56.343284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:56.343734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:56.370933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:56.376518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-af04e651-2af9-4fcb-b7c7-105cb44dfea4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046908 May 19 12:31:56.377130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 917/1956] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:57.160026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:57.160910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:57.168955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:57.169476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:57.170604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:57.171110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:57.234886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:57.243282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bddb93c1-ec38-4dc4-8d9a-cc853071be82 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087795 May 19 12:31:57.243865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1040/1957] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:57 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:57.392245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:57.393849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:57.405178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:57.405654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:57.406394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:57.406827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:57.435918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b682810e-1475-47de-87df-a3748a68520c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050803 May 19 12:31:57.436479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 918/1958] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:58.261039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:58.261680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:58.268682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:58.269015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:58.269762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:58.271195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:58.335902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:58.343996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-673f4c31-0c5d-4cb1-961b-b8c83a9daef5 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.086924 May 19 12:31:58.344563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1041/1959] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:58 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:58.451635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:58.452937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:58.459856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:58.460310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:58.461108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:58.461542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:58.491958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3d2958a9-4456-4e6d-92f0-ec8cb48fa955 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046937 May 19 12:31:58.492827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 919/1960] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:31:59.365139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:59.366063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:59.377341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:59.378069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:59.379287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:59.379987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:59.447101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:31:59.455016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-69104733-9d4c-4e7e-b0dc-63479019307b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096386 May 19 12:31:59.455542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1042/1961] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:31:59 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:31:59.504595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:31:59.505546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:31:59.513310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:59.513836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:59.514687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:31:59.515250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:31:59.546304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-66880a56-2315-48c8-8932-9cdb95a804c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045768 May 19 12:31:59.546858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 920/1962] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:31:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:00.475825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:00.476798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:00.487806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:00.488539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:00.489844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:00.490561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:00.557896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:00.558550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:00.559496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:00.565700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ff61175-ff13-4265-99ce-4bc48e6c6756 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.096285 May 19 12:32:00.566456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1043/1963] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:00 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:00.566587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:00.566940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:00.567789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:00.568221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:00.600410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38374921-8b7c-44cb-9167-aa12c80046e1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045882 May 19 12:32:00.601015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 921/1964] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:01.583043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:01.583657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:01.590976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:01.591411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:01.592170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:01.592679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:01.614337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:01.615364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:01.622454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:01.623230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:01.624028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:01.624428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:01.659332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:01.663859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cde36907-34cc-427c-8ecb-c3376644b081 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053507 May 19 12:32:01.664737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 922/1965] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:01.667575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bc384788-6e5e-42dd-9a79-f59d0a4e948b tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088606 May 19 12:32:01.668214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1044/1966] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:01 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:02.677312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:02.678934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:02.684789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:02.685399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:02.689004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:02.689757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:02.690603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:02.691045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:02.692133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:02.692575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:02.693352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:02.693773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:02.723043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58ca094d-e46c-4a25-b5aa-d768e431f81d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050667 May 19 12:32:02.723936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 923/1967] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:02.758359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:02.766224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6422e3e3-01ea-423c-b0b1-99087beed085 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.085464 May 19 12:32:02.766732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1045/1968] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:02 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:03.737248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:03.738429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:03.748037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:03.748586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:03.749739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:03.750285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:03.781090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-904a8f06-c9bb-402b-a21c-3685223f4c4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049403 May 19 12:32:03.781515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 924/1969] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:03.783478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:03.784223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:03.790941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:03.791377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:03.792111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:03.792526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:03.869411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:03.883198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8711b76b-be1f-4e70-887c-4c755624adb8 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.103525 May 19 12:32:03.884088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1046/1970] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:03 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:04.797531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:04.798478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:04.807101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:04.807559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:04.808703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:04.809249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:04.847964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cf19f3d7-7ca0-4458-b8b5-df899a971a8e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057590 May 19 12:32:04.848525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 925/1971] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:04.907651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:04.908387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:04.919028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:04.919619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:04.920820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:04.921536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:04.927271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:04.986591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:04.994565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b9f42beb-08a0-494a-981e-b2c7073360aa tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093296 May 19 12:32:04.995137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1047/1972] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:04 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:05.842134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:32:05.842351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:32:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:32:05.861064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:05.862017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:05.869748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:05.870192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:05.870996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:05.871457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:05.907532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b028d3d-51d7-4e72-b482-9f75e38098d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050602 May 19 12:32:05.908351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 926/1973] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:06.013799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:06.014526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:06.023787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:06.024229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:06.025121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:06.025566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:06.090433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:06.102077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-31c7f81e-9e09-4bf0-b0d3-bb83f22ac242 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093081 May 19 12:32:06.102703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1048/1974] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:06 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:06.925668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:06.927232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:06.935321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:06.935766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:06.936528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:06.936993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:06.971113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-56efc0ee-148f-4857-bbec-bd5b726e9dc1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052221 May 19 12:32:06.971683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 927/1975] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:07.127612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:07.128571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:07.137394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:07.137837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:07.138585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:07.139011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:07.201880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:07.209563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4de22e9a-0359-49b3-8a6c-22a7047849ac tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088846 May 19 12:32:07.210125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1049/1976] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:07 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:07.983721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:07.984746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:07.992444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:07.992969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:07.993837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:07.994314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:08.025887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aa04204c-1b88-4c30-8354-9aae21c715d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046687 May 19 12:32:08.026441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 928/1977] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:08.235161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:08.236060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:08.245300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:08.245767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:08.246489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:08.246935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:08.321185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:08.329553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f70e2a24-d91e-4d24-b22c-61cb0c3cd89c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.101101 May 19 12:32:08.330124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1050/1978] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:08 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:09.043788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:09.045295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:09.055842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:09.056278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:09.057074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:09.057498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:09.088219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ecb6ab89-496a-42ab-8e9e-dd54b69d5a26 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051178 May 19 12:32:09.088794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 929/1979] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:09.350292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:09.351384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:09.362551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:09.363268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:09.364556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:09.365302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:09.430042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:09.437930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6faea916-5926-490b-89df-e4196daeae40 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094043 May 19 12:32:09.438505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1051/1980] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:09 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:10.105471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:10.106942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:10.117953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:10.118388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:10.119139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:10.119562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:10.148573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9184a3a7-7888-47c7-9d8f-8d41e973257a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050634 May 19 12:32:10.149158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 930/1981] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:10.457447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:10.458114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:10.468164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:10.468898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:10.470062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:10.470684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:10.535757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:10.543711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ccc588be-4f76-487d-aa2b-11b987cd9d5a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092748 May 19 12:32:10.544251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1052/1982] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:10 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:11.160663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:11.161666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:11.169251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:11.169692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:11.170458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:11.170879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:11.200527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3af2ded2-9c67-4f7c-a938-6a311f0e810d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044566 May 19 12:32:11.201125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 931/1983] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:11.560611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:11.561304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:11.567997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:11.568458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:11.569229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:11.569659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:11.632663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:11.640481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0b3b1b6c-fc43-4882-bc62-5ad1649d3a8d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083829 May 19 12:32:11.641082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1053/1984] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:11 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:12.221159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:12.222612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:12.232848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:12.233588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:12.234833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:12.235600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:12.270117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1439efe5-debf-4510-833e-ac318dd7d44f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058216 May 19 12:32:12.270689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 932/1985] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:12.666298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:12.667020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:12.674953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:12.675449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:12.676293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:12.676805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:12.743527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:12.752215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8f60cd69-9134-4e16-936f-e47e800bbc3c tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.092673 May 19 12:32:12.752812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1054/1986] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:12 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:13.283594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:13.284746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:13.292933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:13.293380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:13.294125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:13.294548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:13.324963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c7d7d969-8424-449b-8083-d65de99a31a6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046210 May 19 12:32:13.325437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 933/1987] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:13.602250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:32:13.602581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:32:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:32:13.772748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:13.773733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:13.781523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:13.781999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:13.782742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:13.783200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:13.845658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:13.854072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2cbe3bc9-2a9e-452b-9597-bcfa776d70ec tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087476 May 19 12:32:13.854363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1055/1988] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:13 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:14.107089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:32:14.107367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:32:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:32:14.340106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:14.341184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:14.350197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:14.350759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:14.351645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:14.352156 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:14.373960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:14.387478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-84f8deb4-bf8f-49b6-b9db-b26c8a4113c3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053438 May 19 12:32:14.388037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 934/1989] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:14.875284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:14.876181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:14.886521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:14.886955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:14.887697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:14.888117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:14.953950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:14.961909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a21b5904-1a57-4ddd-88d0-c50a1bdc2b94 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.093686 May 19 12:32:14.962473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1056/1990] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:14 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:15.403536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:15.405049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:15.416126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:15.416866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:15.417666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:15.418094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:15.447971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5745e2ad-dc3f-4690-abaa-2ae90d4b4b4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051200 May 19 12:32:15.448522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 935/1991] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:15.983098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:15.983915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:15.994955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:15.995398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:15.996140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:15.996562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:16.067828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:16.075783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db42c02b-647c-4939-81cc-8e5b8f4cd185 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.099414 May 19 12:32:16.076340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1057/1992] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:15 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:16.463914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:16.465011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:16.473013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:16.473535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:16.474413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:16.474913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:16.510591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a4670d2f-1584-44c2-b6bb-d6432399200d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053446 May 19 12:32:16.511153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 936/1993] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:17.096624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:17.097643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:17.107077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:17.107485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:17.108229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:17.108728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:17.179850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:17.187879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1027d464-afc6-454a-bdb2-3856a809f7ef tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.097740 May 19 12:32:17.188477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1058/1994] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:17 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:17.526509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:17.527460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:17.534362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:17.534804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:17.535559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:17.535983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:17.565272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-236c3230-4bcd-40be-8cc7-94747abeff76 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045319 May 19 12:32:17.565815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 937/1995] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:18.209253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:18.210276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:18.218683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:18.219132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:18.219905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:18.220337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:18.285033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:18.297557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0e22fc08-f65b-41e1-bb20-18c0576c7982 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094732 May 19 12:32:18.298456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1059/1996] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:18 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:18.581974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:18.583451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:18.594020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:18.594760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:18.595985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:18.596718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:18.629995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1e8d4bac-2367-43bb-b400-232756467c9f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054722 May 19 12:32:18.630808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 938/1997] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:19.321039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:19.322082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:19.329406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:19.329845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:19.330592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:19.331045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:19.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:19.409402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c57d564a-d9de-49c0-ba4c-0dd84b703a6a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.094788 May 19 12:32:19.409968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1060/1998] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:19 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:19.646133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:19.647482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:19.655756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:19.656199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:19.656986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:19.657411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:19.686604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5ec96ab2-a0c0-492b-a9e8-f6804f4c24fd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047294 May 19 12:32:19.687224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 939/1999] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:20.438206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:20.438948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:20.449536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:20.450224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:20.451483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:20.452170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:20.521687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:20.530031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f5aa639-8789-4543-be92-933690824269 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.098274 May 19 12:32:20.530597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1061/2000] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:20 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:20.701966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:20.702970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:20.711151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:20.711708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:20.712596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:20.713134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:20.745746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2135f428-5f28-4ac3-bd10-0c0de3df0338 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049646 May 19 12:32:20.746335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 940/2001] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:21.548834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:21.549698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:21.559199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:21.559835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:21.560956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:21.561646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:21.635412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:21.643715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b7746baf-b123-4b2a-83e5-d31f7d2c9537 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.100416 May 19 12:32:21.644289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1062/2002] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:21 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:21.761575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:21.763095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:21.773909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:21.774650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:21.775903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:21.776664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:21.806365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba4d8acd-9445-402e-9bd1-267a766b8138 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051508 May 19 12:32:21.806910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 941/2003] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:22.663803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:22.664438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:22.673219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:22.673664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:22.674407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:22.674834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:22.738245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:22.741707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:22.746643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c4ac925-35b6-42f2-bbb0-cd4c6c6a0380 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.088907 May 19 12:32:22.747212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1063/2004] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:22 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:22.822474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:22.823933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:22.834658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:22.835390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:22.836681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:22.837384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:22.867728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5d858b5d-d123-4318-9dee-019c3ba9f972 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052001 May 19 12:32:22.868259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 942/2005] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:23.768157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:23.769298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:23.780211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:23.781067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:23.781847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:23.782276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:23.857347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:23.869617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-25d7adcf-2921-46d9-b1ae-6a068a403223 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.107945 May 19 12:32:23.870479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1064/2006] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:23 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:23.879970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:23.881227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:23.884894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:23.892294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:23.892831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:23.893964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:23.894536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:23.927269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-592e4cda-95e1-4a43-b53b-93b4602629b2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051328 May 19 12:32:23.927828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 943/2007] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:24.892336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:24.893289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:24.904399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:24.904877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:24.905603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:24.906043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:24.939350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:24.940349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:24.947799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:24.948237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:24.949040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:24.949465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:24.968406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:24.976113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-95282469-fafa-4e9a-b029-94feffe80b33 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.090192 May 19 12:32:24.976910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1065/2008] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:24 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:24.979182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d422eb80-0274-425e-9b7d-386afe89509a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043835 May 19 12:32:24.979738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 944/2009] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:25.995914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:25.997305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:25.998353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:25.998956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:26.001405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=56 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:26.009851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:26.010582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:26.010863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:26.011554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:26.011750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:26.012476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:26.012718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:26.013445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:26.045768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6827acb6-4b5f-4b04-9a6f-e632a8036548 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057172 May 19 12:32:26.046360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1066/2010] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:32:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:32:26.078430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:26.086312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d9f7fc52-e086-4574-98f1-89efa89cb562 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095470 May 19 12:32:26.086881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 945/2011] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:25 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:26.785833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:32:26.786214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:32:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:32:27.103168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:27.103758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:27.110507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:27.110954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:27.111713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:27.112139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:27.174549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:27.182321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-40783628-f1d2-4416-be69-5f7616b5561a tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.083066 May 19 12:32:27.182966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 946/2012] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:27 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:27.188465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:32:27.189410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:27.195486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:27.195920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:27.196664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:27.197106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:27.299303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7ca7cee9-8045-4887-a9bd-c4d26ec35435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action" status: 200 len: 76246 microversion: 2.1 time: 0.243978 May 19 12:32:27.299975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1067/2013] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:32:27 2022] POST /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1/action => generated 76246 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:32:28.203431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:28.204450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:28.212227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:28.212757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:28.213593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:28.214097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:28.276808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:28.284460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15f2f4f3-aa1b-40e4-a6e0-4286cd7c3f5d tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.087713 May 19 12:32:28.285060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 947/2014] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:28 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:29.305566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:29.306731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] [instance: 08f420db-5800-43e3-9990-bd7aa43a9d3e] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:29.316912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:29.317356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:29.318115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:29.318542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:29.385361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f8e5131aab4e43a18d98e719e83d6268', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1082ac24dda74201ab7a9bf670d83a38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:29.394065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ccedc23c-6059-4436-a2fe-85b6ced891ea tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e" status: 200 len: 1219 microversion: 2.1 time: 0.095638 May 19 12:32:29.394649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1068/2015] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:29 2022] GET /compute/v2.1/servers/08f420db-5800-43e3-9990-bd7aa43a9d3e => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:29.573638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e4a0afb2-878e-4c2f-a84f-0bcdf3f34923 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] Calling method '.version_select at 0x7fe1fff7b9d0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:29.598349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e4a0afb2-878e-4c2f-a84f-0bcdf3f34923 tempest-TestShelveInstance-1210100504 tempest-TestShelveInstance-1210100504-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-246156379" status: 202 len: 0 microversion: 2.1 time: 0.028639 May 19 12:32:29.600271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 948/2016] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:32:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-246156379 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 19 12:32:31.266927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:31.268004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:31.274809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.275432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.276656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.277244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.304311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:32:31.372878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:31.374597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:31.377308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:32:31.377462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:32:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 (ip 10.222.0.35) !!! May 19 12:32:31.389917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.390629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.391686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.392327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.415818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f8199e98-c510-4d9f-86f1-8b87ff8f88ea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 204 len: 0 microversion: 2.1 time: 0.152434 May 19 12:32:31.427749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1069/2017] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:32:31 2022] DELETE /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:32:31.438357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:31.439513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:31.442915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:32:31.449046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.449500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.450320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.450751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.549698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0014fd58-4c2b-4105-8d9b-ea549d931404 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 204 len: 0 microversion: 2.20 time: 0.183797 May 19 12:32:31.550574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 949/2018] 10.222.0.35 () {60 vars in 1279 bytes} [Thu May 19 12:32:31 2022] DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 19 12:32:31.559814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:31.561682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:31.568862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.569640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.571595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:31.572847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:31.598288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:31.604360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:31.618353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a1c38646-4922-4cb2-adf6-727d61667556 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.185942 May 19 12:32:31.621016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1070/2019] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:31 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:31.720809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d6997ba-104f-44b3-b826-b5aa21983198 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] Calling method '.version_select at 0x7fe1fffa1700>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:31.745515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d6997ba-104f-44b3-b826-b5aa21983198 tempest-TestVolumeBootPattern-1824824684 tempest-TestVolumeBootPattern-1824824684-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-150760622" status: 202 len: 0 microversion: 2.1 time: 0.028482 May 19 12:32:31.746098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1071/2020] 10.222.0.35 () {58 vars in 1259 bytes} [Thu May 19 12:32:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-150760622 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:32:31.853154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:31.864362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:31.865882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:31.867069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8006c320-48d9-46ec-96e5-bcc8d33c8a98 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.312082 May 19 12:32:31.867624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 950/2021] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:32.636887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:32.637554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:32.650849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:32.651293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:32.652083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:32.652528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:32.892496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:32.893494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:32.903155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:32.903688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:32.904519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:32.904974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:32.987531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:33.000778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2133ff4f-6a65-4aa6-bb11-b28407d9d245 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.368551 May 19 12:32:33.001471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1072/2022] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:32:33.185545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:33.196887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:33.199095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:33.202296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-81cff83b-a1f0-4c6e-8419-42e279b861ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.317850 May 19 12:32:33.203678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 951/2023] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:34.023331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:34.023913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:34.046833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:34.047274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:34.052142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:34.052589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:34.220716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:34.222054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:34.228858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:34.230389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:34.230869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:34.231526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:34.294849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:34.304394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cab19b0d-beaa-4e40-99f6-b07b613ea13b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.284770 May 19 12:32:34.305716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1073/2024] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:34.455885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:34.467436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:34.469740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:34.471473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0d6dbaec-d635-454a-97ef-f7c444508bbf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.254670 May 19 12:32:34.473247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 952/2025] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:35.321944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:35.323416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:35.330683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:35.331865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:35.332098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:35.332533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:35.490052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:35.491276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:35.502438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:35.503130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:35.503897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:35.504320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:35.775243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:35.787340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:35.788866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:35.790055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2c8baa47-c3ac-4fbe-aba9-85b217150fe4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.304054 May 19 12:32:35.790622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 953/2026] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:35.862836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:35.889507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c6391303-2181-47f6-8d5f-97bbd893ac3a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.571277 May 19 12:32:35.891373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1074/2027] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:35 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:36.808117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:36.809346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:36.817395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:36.818218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:36.819131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:36.819737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:36.907436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:36.908144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:36.924160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:36.925130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:36.927418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:36.928276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:37.104571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:37.116297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-70373bb1-9a28-4111-be93-030491ab20c4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.212673 May 19 12:32:37.116937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1075/2028] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:37.145805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:37.154249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:37.155747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:37.157138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e1aaaf06-b317-4404-9efd-5bd7ed57247c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.353448 May 19 12:32:37.158284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 954/2029] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:38.136347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:38.137521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:38.144553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:38.145029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:38.145887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:38.146393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:38.175159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:38.175745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:38.182301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:38.182998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:38.184201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:38.184835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:38.336945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:38.366647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5aabeb2-e47e-470f-8c2a-4a7856a0b990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.234021 May 19 12:32:38.385271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1076/2030] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:38 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:38.729241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:38.738431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:38.740240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:38.741599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c3dfb09d-b4ae-47c8-ac81-8255a4d21bd3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.570267 May 19 12:32:38.742438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 955/2031] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:32:39.385238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:39.386615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:39.398145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:39.398578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:39.400958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:39.401432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:39.663955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:39.673168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-79ea0f3a-584f-4caf-88cb-2cf15110a322 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.293396 May 19 12:32:39.673744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1077/2032] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:32:39.760021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:39.760720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:39.767779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:39.768305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:39.769168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:39.769667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:39.937836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:39.953280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:39.954816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:39.956085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-41169c17-92aa-4825-9dfb-b1eb7ac9fbdb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.200105 May 19 12:32:39.957045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 956/2033] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:40.690634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:40.691848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:40.698567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:40.699082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:40.699860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:40.700259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:40.975138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:40.976461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:40.986198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:40.986705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:40.987693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:40.988186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:41.123576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:41.136330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-34b725e5-fead-4705-a6b2-693aa6da78b1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.449363 May 19 12:32:41.149582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1078/2034] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:41.317512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:41.326400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:41.327895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:41.329098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ecb8bc0-1845-4bf5-83d2-1b1dec11b6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.358369 May 19 12:32:41.329683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 957/2035] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:32:42.160492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:42.161558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:42.172579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:42.173354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:42.174573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:42.175292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:42.322293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:42.330726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9c1e2a84-7e47-404f-ae38-7fc3c2098bd7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176881 May 19 12:32:42.331299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1079/2036] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:42 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:42.347796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:42.348424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:42.355189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:42.355663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:42.356406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:42.356859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:42.492791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:42.502599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:42.504065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:42.505261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b04198c1-cf60-4ad1-b259-3467c82550a7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161777 May 19 12:32:42.505816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 958/2037] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:43.349728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:43.350421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:43.357970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:43.358401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:43.359142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:43.359557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:43.363668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:43.503775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:43.514779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a4d83115-8814-4c4a-bf49-207ddc5991d0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168684 May 19 12:32:43.515503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1080/2038] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:43 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:43.523304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:43.523930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:43.533378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:43.533815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:43.534594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:43.535010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:43.677026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:43.685746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:43.687291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:43.688444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a125d801-8b61-4ec3-8547-01be058da488 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.168965 May 19 12:32:43.689231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 959/2039] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:44.531656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:44.532272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:44.539269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:44.539703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:44.540447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:44.540911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:44.685156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:44.693796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6e3cc822-1c60-4f30-a9fd-920e4afce6f8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166079 May 19 12:32:44.694386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1081/2040] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:44 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:44.707149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:44.707726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:44.714572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:44.715011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:44.715761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:44.716182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:44.869165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:44.879914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:44.881436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:44.882603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-297e9771-3e54-4109-80c8-92ec81b24928 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.179479 May 19 12:32:44.883426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 960/2041] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:45.715833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:45.716551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:45.724187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:45.724655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:45.725417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:45.725847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:45.869364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:45.878053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a3719469-789a-4ab0-8973-de0b9e13ae77 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167721 May 19 12:32:45.878623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1082/2042] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:45 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:45.900933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:45.901492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:45.908454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:45.908912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:45.909653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:45.910073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:46.047644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:46.056449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:46.057968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:46.059129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-944ab730-eb45-4d65-9eae-16c2d07506cc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162112 May 19 12:32:46.059685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 961/2043] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:46.896690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:46.897590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:46.908827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:46.909597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:46.910763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:46.911507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:47.052683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:47.061154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b33854bf-20db-4902-a121-439268950c31 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169080 May 19 12:32:47.061832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1083/2044] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:47.077714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:47.078381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:47.085147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:47.085580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:47.086320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:47.086740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:47.233998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:47.244232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:47.245796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:47.246978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-76b379b3-9c68-4204-b32f-05cacef5e698 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.173147 May 19 12:32:47.247531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 962/2045] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:32:48.082649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:48.083687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:48.094653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:48.095452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:48.096192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:48.096611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:48.244820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:48.253567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b0f0f1e6-906f-42df-8740-a623cbc639db tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.177346 May 19 12:32:48.254209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1084/2046] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:48 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:48.267096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:48.267738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:48.274122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:48.274560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:48.275637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:48.276066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:48.409562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:48.418566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:48.420034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:48.421225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fffdb4df-b593-4f92-9fa2-4a7c2a281c66 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.159470 May 19 12:32:48.421764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 963/2047] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:49.275474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:49.276296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:49.286311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:49.286755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:49.287506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:49.287929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:49.416960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:49.425002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c35862f-3efd-4ae9-b60f-59a956c4c280 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156303 May 19 12:32:49.425553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1085/2048] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:49.439261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:49.439848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:49.446585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:49.447026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:49.447768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:49.448191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:49.596203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:49.605328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:49.606883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:49.608055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f623dea3-fc6f-4d67-a120-d8be8d47f5c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.172700 May 19 12:32:49.608616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 964/2049] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:50.446159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:50.447106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:50.457559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:50.457996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:50.458752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:50.459176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:50.602888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:50.614414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eccf7d9d-c0a6-4d20-98d0-eeafe392a29d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.174718 May 19 12:32:50.614986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1086/2050] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:50 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:50.626618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:50.627221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:50.634263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:50.634693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:50.635438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:50.635860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:50.770842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:50.779769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:50.781273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:50.782432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa066c22-9ccf-46e4-a86b-1ab937c1f2cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160001 May 19 12:32:50.782984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 965/2051] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:51.634137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:51.634849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:51.644228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:51.644821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:51.645891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:51.646426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:51.780554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:51.789237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d80d4df0-e774-4898-b6d8-98a911b4b8d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160019 May 19 12:32:51.789839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1087/2052] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:51 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:51.800680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:51.801299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:51.807992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:51.809443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:51.810829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:51.811261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:51.955635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:51.964434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:51.965946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:51.967100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-09042f41-0000-4701-97e4-435f2a9a8531 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170418 May 19 12:32:51.967632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 966/2053] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:52.811150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:52.812143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:52.823357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:52.824145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:52.825433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:52.826142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:52.971893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:52.982879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-05784d95-ba17-4ec9-ae1e-4ed0608e3c11 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178347 May 19 12:32:52.983432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1088/2054] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:52.985740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:52.986338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:52.993510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:52.993955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:52.994721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:52.995147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:52.999517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:53.137113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:53.145917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:53.147475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:53.148763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3773ba65-8b73-4fc4-b125-f709a4bf343a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166828 May 19 12:32:53.149322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 967/2055] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:32:54.004249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:54.004947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:54.012684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:54.013184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:54.014003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:54.014470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:54.156552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:54.166382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-75c9792c-5f69-463e-a0d4-57f2e3f473c5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168764 May 19 12:32:54.166952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1089/2056] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:53 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:54.167211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:54.167811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:54.170126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=56,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:54.175509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:54.175955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:54.176724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:54.177147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:54.312375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:54.326010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:54.328329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:54.330170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1ab5fa2c-1f09-479c-9769-03432bc7fdb3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166636 May 19 12:32:54.331003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 968/2057] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:32:55.187740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:55.188729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:55.199905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:55.200662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:55.201924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:55.202633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:55.340833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:55.349087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:55.349626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-337a7d36-bc44-46ea-a654-dc5051dc8486 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168355 May 19 12:32:55.349769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:55.350245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1090/2058] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:55.356611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:55.357069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:55.357853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:55.358281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:55.499354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:55.508371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:55.509970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:55.511145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-92526e1b-6291-4add-80e6-c41596a23aac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166130 May 19 12:32:55.511708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 969/2059] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:56.367232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:56.367851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:56.370322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:56.376194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:56.376658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:56.377408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:56.377838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:56.514340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:56.522567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ec4ccdeb-89f7-488f-90d0-531e20ef029a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159217 May 19 12:32:56.523167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1091/2060] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:56 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:56.529313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:56.529915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:56.536812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:56.537260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:56.538011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:56.538434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:56.677620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:56.688925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:56.691270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:56.692971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9002023d-e2e3-46b1-9d32-c9d786b1753a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.167265 May 19 12:32:56.693598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 970/2061] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:57.541144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:57.541933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:57.551330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:57.551909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:57.552934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:57.553493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:57.695283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:57.707325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-26a5cd6d-7b9e-4294-b763-54a90f20f4da tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171305 May 19 12:32:57.707882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1092/2062] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:57.711473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:57.712261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:57.719152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:57.719587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:57.720399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:57.720856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:57.848886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:57.857489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:57.859034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:57.860177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2f1dcd6e-ab74-4c10-83a1-0fbbf105182b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.152561 May 19 12:32:57.860759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 971/2063] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:58.727347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:58.728403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:58.739123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:58.739567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:58.740310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:58.740765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:58.877349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:58.878544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:58.879154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:58.885748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ce5dac9d-0901-44ae-ab54-28941262bb23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164909 May 19 12:32:58.885972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:58.886318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1093/2064] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:32:58.886488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:58.887265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:58.887697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:59.055477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:59.067370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:59.068900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:32:59.070072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96992ea0-b611-4bd1-9b7c-5d334abf4e96 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.195477 May 19 12:32:59.070655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 972/2065] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:32:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:32:59.907184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:32:59.908422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:32:59.915973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:59.916444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:32:59.917212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:32:59.917631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:00.051717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:00.064049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b320027c-670d-4529-badd-e90ba938da36 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163023 May 19 12:33:00.064996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1094/2066] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:32:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:00.088780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:00.089421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:00.096145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:00.096597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:00.097388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:00.097855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:00.240240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:00.249453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:00.250951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:00.252127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a7c008e-c6c6-4c11-bc1c-63bdc8140400 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.167356 May 19 12:33:00.252688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 973/2067] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:01.084137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:01.084784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:01.091883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:01.092320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:01.093093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:01.093533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:01.242740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:01.255422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-61495dc8-6772-42b7-8e0b-d8f48e1e0490 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176167 May 19 12:33:01.256051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1095/2068] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:01.270979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:01.271618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:01.278332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:01.278798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:01.279553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:01.279980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:01.422274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:01.435321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:01.437775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:01.439535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dd35b0e2-5c79-4465-98ff-4f1111613b2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.172298 May 19 12:33:01.440319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 974/2069] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:02.277267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:02.278260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:02.288033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:02.288474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:02.289246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:02.289666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:02.423602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:02.427837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:02.433423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aff213ff-e610-4c70-bb95-018f37c61c57 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162649 May 19 12:33:02.433971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1096/2070] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:02.460517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:02.461166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:02.467749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:02.468194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:02.468994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:02.469423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:02.605385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:02.614224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:02.615717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:02.616934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b370d9d-f24c-48dd-8b1b-490ac0adb98b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161114 May 19 12:33:02.617472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 975/2071] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:03.454360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:03.455325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:03.463274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:03.463705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:03.464451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:03.464969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:03.602630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:03.610712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1dab2b6b-0496-4ab6-a71e-f44da0ad1fc9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162930 May 19 12:33:03.611307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1097/2072] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:03 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:03.635510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:03.636059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:03.643043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:03.643482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:03.644234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:03.644677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:03.777463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:03.786470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:03.787945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:03.789614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-403e766c-3203-4d07-9d9e-1b7ef98cf7c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157603 May 19 12:33:03.789829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 976/2073] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:04.632407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:04.633412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:04.644814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:04.645293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:04.646031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:04.646454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:04.793674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:04.801907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-60e56dc2-108d-4868-81ad-3fd733f39b89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176020 May 19 12:33:04.802477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1098/2074] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:04 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:04.812897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:04.813541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:04.820380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:04.820870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:04.821624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:04.822054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:04.959816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:04.968844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:04.970338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:04.971509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6b3c05cf-b68f-4e75-b4c1-dacad0390d58 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162541 May 19 12:33:04.972070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 977/2075] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:05.821135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:05.821750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:05.828992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:05.829419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:05.830198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:05.830605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:05.962289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:05.973851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2a713e86-f453-42a1-8dd5-0b9879a4f8b0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156988 May 19 12:33:05.974415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1099/2076] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:05.989667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:05.990251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:05.997502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:05.997946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:05.999053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:05.999460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:06.131142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:06.139617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:06.141144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:06.142308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2a641446-efb8-4823-a4f8-6be6dae740fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156540 May 19 12:33:06.142900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 978/2077] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:06.995651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:06.996677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:07.012120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:07.014500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:07.015751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:07.016484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:07.165075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:07.168952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:07.169542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:07.174353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c6ebf870-34cf-4547-a46d-4b896564eb59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.185298 May 19 12:33:07.174998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1100/2078] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:07.176604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:07.177084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:07.177850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:07.178273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:07.324506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:07.337517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:07.339874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:07.341630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-900d262d-9aae-404d-98c0-3e488f7133cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.176383 May 19 12:33:07.342450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 979/2079] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:08.195662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:08.196588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:08.207184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:08.207908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:08.209188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:08.209927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:08.359345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:08.363969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:08.364573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:08.369119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-80b84897-a5f9-47aa-a129-3ac64602696f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.179982 May 19 12:33:08.369688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1101/2080] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:08.372188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:08.372678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:08.373843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:08.374286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:08.505893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:08.518175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:08.520392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:08.522159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dc942b31-540a-461a-bd98-e242d3e5ec23 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.163472 May 19 12:33:08.522714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 980/2081] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:09.391741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:09.392773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:09.401263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:09.401703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:09.402464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:09.402885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:09.539932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:09.541267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:09.541872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:09.548223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6608bb91-5aa3-4176-9e6e-a288018fcfd3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163733 May 19 12:33:09.548922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:09.549099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1102/2082] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:09 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:09.549373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:09.550137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:09.550593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:09.687056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:09.695755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:09.697282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:09.698446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ffe816d3-5091-4abd-be17-5f6013e6b59b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161892 May 19 12:33:09.699013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 981/2083] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:10.569463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:10.570126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:10.577851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:10.578358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:10.579203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:10.579682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:10.710328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:10.716607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:10.717224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:10.718860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c961997c-b9ed-4380-9dc3-4589f500584e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155492 May 19 12:33:10.719581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1103/2084] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:10.723732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:10.724169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:10.725003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:10.725424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:10.862584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:10.871656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:10.873482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:10.874677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1b3d13ee-beb5-4e56-b090-daf386a4a70a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161910 May 19 12:33:10.875243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 982/2085] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:11.739537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:11.740502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:11.752044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:11.752816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:11.754057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:11.754770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:11.893241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:11.893859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:11.898727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:11.900531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:11.900997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:11.901761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:11.902184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:11.906794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c4df776d-6176-4ac5-8a50-62eb28864fa8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.173865 May 19 12:33:11.907366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1104/2086] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:12.028277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:12.031663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:12.036887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:12.038459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:12.039619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e241a0d5-7ecc-45d7-bc25-dce96691345d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.150448 May 19 12:33:12.040188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 983/2087] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:12.928359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:12.929343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:12.940836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:12.941565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:12.942787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:12.943484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:13.063764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:13.064349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:13.072211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:13.073175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:13.073872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:13.074454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:13.090359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:13.099234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-845465b6-3710-49f2-9bc3-501ee852f16f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.177682 May 19 12:33:13.100197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1105/2088] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:13.214929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:13.227207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:13.228692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:13.229855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e6719440-3c7b-47b8-9501-e0d5f60ab533 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170423 May 19 12:33:13.230441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 984/2089] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:14.121192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:14.122205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:14.132875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:14.133317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:14.134070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:14.134496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:14.248658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:14.249438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:14.256404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:14.256881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:14.257613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:14.258274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:14.269512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:14.278041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f5efcba6-f6b8-486c-b2b4-9732e902d27d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163530 May 19 12:33:14.278605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1106/2090] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:14.391047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:14.401722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:14.403257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:14.404431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-355c64b8-bac0-4d9a-9cb2-501adc57abb6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.159961 May 19 12:33:14.405030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 985/2091] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:15.299740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:15.300969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:15.308859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:15.309297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:15.310050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:15.310471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:15.425396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:15.426680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:15.433902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:15.434612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:15.435500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:15.435947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:15.441516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:15.449628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e0780397-f84e-4e2e-9f90-71caf84cf761 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156380 May 19 12:33:15.450203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1107/2092] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:15.573415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:15.582508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:15.583989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:15.585221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-630cf17a-5236-43a0-bfd7-4c3256fe5e10 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.163742 May 19 12:33:15.585781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 986/2093] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:16.470896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:16.471864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:16.480325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:16.480804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:16.481553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:16.481979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:16.603503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:16.604091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:16.611102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:16.611271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:16.611543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:16.612303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:16.612818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:16.619883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-41f92f96-9f0a-4d92-9e3d-752f18ee78c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155409 May 19 12:33:16.620459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1108/2094] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:16.753273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:16.766335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:16.768514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:16.770374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-faa0ac0e-3982-4f02-97a9-741714c6225d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170476 May 19 12:33:16.771167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 987/2095] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:17.641891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:17.642824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:17.651265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:17.651721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:17.652464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:17.652920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:17.789644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:17.791117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:17.792463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:17.797648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:17.798332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:17.799237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:17.799688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:17.801415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a8266213-caae-42be-8b7c-d74734d26860 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166225 May 19 12:33:17.802017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1109/2096] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:17.932604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:17.941087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:17.942570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:17.943830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-71ecb677-0214-47fd-ac19-29dc18fa1d21 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.158193 May 19 12:33:17.944402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 988/2097] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:18.819600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:18.820272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:18.827328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:18.827770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:18.828521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:18.828973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:18.961923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:18.962517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:18.965045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:18.969043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:18.969486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:18.970243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:18.970670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:18.973116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-137a256c-c0a1-4190-afdf-7654ae99cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157523 May 19 12:33:18.973666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1110/2098] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:18 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:19.106868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:19.115285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:19.120699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:19.122820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58efc622-8f20-4518-89f1-ec4416760305 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.164119 May 19 12:33:19.123721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 989/2099] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:19.994762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:19.995659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:20.007173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:20.007618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:20.008366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:20.008821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:20.141225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:20.142524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:20.143127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:20.149505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a9de6d7-eae3-4ccd-8e99-6f31e62b9a03 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161371 May 19 12:33:20.149920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:20.150367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1111/2100] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:20.150498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:20.151125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:20.151551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:20.289126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:20.301706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:20.303849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:20.305540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1f76d6fd-ed7e-4891-9e9d-0b65f2df1a49 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166634 May 19 12:33:20.306256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 990/2101] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:21.166729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:21.167445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:21.175997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:21.176486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:21.177374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:21.177850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:21.329129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:21.329735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:21.330668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:21.336448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:21.336940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:21.337703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:21.338132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:21.338803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-201084e9-07ed-482a-b708-c5afed943ae4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176436 May 19 12:33:21.339372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1112/2102] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:21.486477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:21.496316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:21.498724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:21.501032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afdc0cb7-40f2-4135-97cd-bc1b11e1f30e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.176193 May 19 12:33:21.502154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 991/2103] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:22.359429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:22.360336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:22.370117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:22.370782 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:22.371829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:22.372466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:22.394603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:22.515426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:22.519472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:22.520081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:22.524361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f50467df-245b-4591-a98d-9b039d6938f0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171303 May 19 12:33:22.524972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1113/2104] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:22 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:22.526828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:22.527273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:22.528029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:22.528454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:22.670109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:22.681684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:22.683175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:22.684350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b9bf491a-8c96-462f-b26f-2c9f74808b97 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.168745 May 19 12:33:22.684941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 992/2105] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:23.541887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:23.542493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:23.549442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:23.549886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:23.550632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:23.551056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:23.687477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:23.697168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a65ba492-81ab-4dfa-9f92-946782437ed4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159203 May 19 12:33:23.697709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1114/2106] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:23 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:23.702738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:23.703348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:23.710625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:23.711071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:23.711830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:23.712264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:23.856684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:23.866321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:23.867810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:23.869019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-376b3109-7639-483f-a111-61a8c4ecc87d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170155 May 19 12:33:23.869612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 993/2107] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:24.719017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:24.719969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:24.729947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:24.730386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:24.731132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:24.731553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:24.872854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:24.881546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86910912-8c39-4690-8ef8-f41a0526d2c9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169245 May 19 12:33:24.883617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1115/2108] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:24 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:24.887381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:24.887986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:24.891603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:24.896036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:24.896690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:24.897576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:24.897999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:25.032354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:25.044558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:25.046076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:25.047273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9c8fc8e9-1e8b-4b54-ab8d-658e32265f9e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.163773 May 19 12:33:25.047839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 994/2109] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:25.901136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:25.901819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:25.909861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:25.910354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:25.911186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:25.911663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:26.057096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:26.066220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-426dac21-b4b9-49e8-bac1-7b13a071eac0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169931 May 19 12:33:26.066402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:26.066872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1116/2110] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:25 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:26.066977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:26.073650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:26.074091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:26.074848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:26.075273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:26.217135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:26.226160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:26.227644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:26.228830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dbf71b12-816e-4620-bc74-dad975a65922 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166465 May 19 12:33:26.229385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 995/2111] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:27.086565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:27.087192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:27.089726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=52 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:27.095779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:27.096252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:27.097052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:27.097511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:27.248111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:27.249092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:27.256444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:27.256931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:27.257710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:27.258134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:27.398965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:27.408304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:27.409831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:27.411019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-768aeddf-44cd-4975-b983-1b0a4456f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.167406 May 19 12:33:27.411582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 996/2112] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:27.497159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:27.505754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c3f7f1f-1d5b-4080-9929-42035c1dee00 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.424785 May 19 12:33:27.506739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1117/2113] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:28.433538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:28.434507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:28.445175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:28.445626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:28.446386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:28.446818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:28.523449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:28.524096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:28.531106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:28.531535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:28.532278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:28.532731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:28.578128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:28.587106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:28.588581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:28.589795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7dbf92e1-0c1a-4c94-9695-c6a5cba78364 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162892 May 19 12:33:28.590350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 997/2114] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:28.661698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:28.674375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c1ffd1c1-1844-4609-b1d6-6839af4e1b8f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154522 May 19 12:33:28.675252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1118/2115] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:28 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:29.611882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:29.612962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:29.620962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:29.621402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:29.622148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:29.622570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:29.692675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:29.693295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:29.699958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:29.700566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:29.701145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:29.702737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:29.755112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:29.763811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:29.766003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:29.767312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-581e6117-ddd9-4365-ba54-61a457bc66b0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161948 May 19 12:33:29.767989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 998/2116] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:29.837888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:29.845977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-75592a06-0907-4212-abc2-3f07ff764dc7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157504 May 19 12:33:29.846553 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1119/2117] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:29 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:33:30.789571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:30.790275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:30.799471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:30.800064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:30.801121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:30.801699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:30.863153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:30.863784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:30.870805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:30.871308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:30.872069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:30.872493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:30.940240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:30.948902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:30.950390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:30.951527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c0af1316-2cda-4b51-8cab-bbe9407b9b12 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.168389 May 19 12:33:30.952089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 999/2118] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:31.003523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:31.011748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-67ab1a19-5bc6-4805-bb98-a1187062f24d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.152563 May 19 12:33:31.012324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1120/2119] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:30 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:31.975254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:31.975831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:31.982736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:31.983170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:31.983911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:31.984631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:32.029232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:32.029851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:32.037259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:32.037741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:32.038506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:32.038931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:32.166562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:32.183784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:32.192039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8eb312d4-3758-42e6-948a-907775fe0225 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166707 May 19 12:33:32.192671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1121/2120] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:32.287824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:32.296621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:32.298146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:32.299321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-90b3c24a-9573-4aa9-9a2d-00750b7126e0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.330653 May 19 12:33:32.299866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1000/2121] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:33.209078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:33.209656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:33.216805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:33.217246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:33.217999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:33.218423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:33.316980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:33.317948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:33.327018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:33.327566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:33.328758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:33.329302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:33.362387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:33.370674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee71a212-0201-4775-8394-92497bde7a8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165454 May 19 12:33:33.371341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1122/2122] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:33 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:33.469818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:33.480869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:33.482351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:33.483518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ebbd4680-1439-424e-bc56-4d8727756e0b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170572 May 19 12:33:33.484053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1001/2123] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:34.392041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:34.393017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:34.405363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:34.406085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:34.407301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:34.408001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:34.502140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:34.502767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:34.509722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:34.510164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:34.510927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:34.511362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:34.544944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:34.553167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aae84e25-a8da-4a65-9cbe-f25f9544a7a4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167584 May 19 12:33:34.553750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1123/2124] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:34.640611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:34.649764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:34.651151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:34.652346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-21b974b4-c6de-4cc0-a630-c857f8c46157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154538 May 19 12:33:34.652946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1002/2125] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:35.574901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:35.575917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:35.587090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:35.587534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:35.588291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:35.588750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:35.670695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:35.671297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:35.677880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:35.678316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:35.679069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:35.679793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:35.734613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:35.742727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18ee97f4-e10b-4e2d-8680-3e9f1dff4c4e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.174442 May 19 12:33:35.743291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1124/2126] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:35 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:35.810197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:35.818781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:35.820365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:35.821560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d0eddf3d-abd7-4ef1-8ed4-6a666ae7f985 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154778 May 19 12:33:35.822111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1003/2127] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:36.764598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:36.765449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:36.776488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:36.777270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:36.778504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:36.779212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:36.840181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:36.840820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:36.847585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:36.848020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:36.848813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:36.849235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:36.915359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:36.923345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e5b3df89-8312-4c33-9b9e-1e8c73d53152 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165280 May 19 12:33:36.923951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1125/2128] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:36.981488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:36.993616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:36.995112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:36.996293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3691660d-2c2f-4a66-b374-d042c5ded110 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160154 May 19 12:33:36.996899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1004/2129] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:37.945258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:37.945897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:37.952909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:37.953346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:37.954083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:37.954526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:38.015268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:38.015839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:38.023458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:38.023630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:38.024364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:38.024833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:38.089179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:38.097399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ddd0f734-3efe-4f8e-8493-b9e2edc55625 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159283 May 19 12:33:38.097939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1126/2130] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:37 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:38.160307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:38.170147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:38.171658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:38.172873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9811b671-94a4-44ce-ac0e-23a1b69c9ac2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161488 May 19 12:33:38.173423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1005/2131] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:39.119087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:39.120010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:39.131487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:39.132238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:39.133541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:39.134250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:39.191638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:39.192288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:39.199016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:39.199570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:39.200312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:39.200764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:39.270030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:39.278116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-27913306-8d69-4656-a94f-23d154e4302b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165565 May 19 12:33:39.278694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1127/2132] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:39.331186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:39.339823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:39.341347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:39.342513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-30d844ec-64b5-4453-85df-872403ab6f0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154884 May 19 12:33:39.343068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1006/2133] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:40.299505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:40.300320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:40.311246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:40.311990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:40.313272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:40.313980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:40.360984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:40.361574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:40.368311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:40.368796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:40.369556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:40.369982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:40.447545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:40.455836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-428cae55-62f2-4aab-a1c0-56f14e1b4aba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162844 May 19 12:33:40.456454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1128/2134] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:40.499963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:40.508679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:40.510187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:40.511351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42b452a0-ddc7-4750-a543-668716dd325c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154326 May 19 12:33:40.511904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1007/2135] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:33:41.476439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:41.477440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:41.488537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:41.489323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:41.490563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:41.491269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:41.529713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:41.530403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:41.537430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:41.537958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:41.538797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:41.539286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:41.627444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:41.635644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a3bd995-374f-47de-a805-b2fe10aa9188 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165699 May 19 12:33:41.636225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1129/2136] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:41 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:41.678694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:41.687362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:41.688910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:41.690078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b4df1a91-6ab6-4ccb-ae80-bda22eb2b590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.164376 May 19 12:33:41.690676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1008/2137] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:42.657197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:42.658155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:42.667314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:42.667759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:42.668503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:42.668964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:42.672967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:42.708745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:42.709354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:42.716356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:42.716833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:42.717592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:42.717990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:42.801426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:42.809572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bf1c8dd5-5e99-4c0d-a4cc-c21937956ba7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158878 May 19 12:33:42.810157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1130/2138] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:42 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:42.847648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:42.856227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:42.857697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:42.858896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-96d1ab61-a87a-4a4e-8b68-b6b971090bd5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154071 May 19 12:33:42.859457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1009/2139] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:43.829313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:43.830104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:43.838293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:43.838740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:43.839495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:43.839906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:43.877959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:43.878626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:43.885427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:43.885953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:43.886773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:43.887266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:43.972395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:43.981173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be653398-f64a-4623-970c-14cf57680479 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156702 May 19 12:33:43.981808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1131/2140] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:43 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:44.019453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:44.027915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:44.029430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:44.030620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9d6a4817-cb11-41a3-8907-b196385cbe69 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157246 May 19 12:33:44.031173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1010/2141] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:45.003229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:45.004301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:45.012578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:45.013054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:45.013806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:45.014232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:45.049172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:45.049827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:45.056917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:45.057366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:45.058137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:45.058576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:45.146810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:45.155071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2e5f88b1-8c84-46d2-9018-4dea6d6f4fb5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158971 May 19 12:33:45.155889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1132/2142] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:44 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:45.189582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:45.198124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:45.199601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:45.200851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8db5f6bc-5c13-4c06-8d75-fec48254d4ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.155625 May 19 12:33:45.201407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1011/2143] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:46.175799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:46.176388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:46.185053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:46.185609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:46.186370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:46.186825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:46.219234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:46.219815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:46.226878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:46.227329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:46.228076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:46.228504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:46.321851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:46.330127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c01352f6-299c-43d8-9022-d929a531e853 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160815 May 19 12:33:46.330697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1133/2144] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:46.358587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:46.367530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:46.369037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:46.370208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cc5aee09-1a7e-44f4-a7fc-ae6532d29295 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154933 May 19 12:33:46.370761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1012/2145] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:47.351490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:47.352465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:47.363505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:47.364183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:47.365453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:47.366150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:47.389117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:47.389687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:47.396309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:47.396759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:47.397549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:47.397979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:47.505778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:47.514231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac78faa7-3f4c-426e-92e1-7be4966d696c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169364 May 19 12:33:47.514841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1134/2146] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:47 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:47.544566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:47.553171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:47.554774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:47.556026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01c6d577-a11c-4e91-8900-98f67003638b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.170938 May 19 12:33:47.556598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1013/2147] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:48.531596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:48.532279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:48.539710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:48.540144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:48.540935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:48.541361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:48.574569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:48.575169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:48.581767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:48.582213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:48.583010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:48.583438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:48.674202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:48.682434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c275c1d-c3ae-4b63-a2f9-a27433eea598 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154784 May 19 12:33:48.683017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1135/2148] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:48 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:48.715915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:48.724479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:48.726003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:48.727272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01bf4dfb-c1c0-4a7e-888b-e0b1328b6a41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156644 May 19 12:33:48.727874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1014/2149] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:49.703437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:49.704400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:49.713793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:49.714233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:49.714981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:49.715404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:49.745997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:49.746676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:49.753920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:49.754358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:49.755104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:49.755502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:49.850510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:49.858811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-73ee9e14-9ace-4bab-bb83-0f9b67dd3c2b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161949 May 19 12:33:49.859385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1136/2150] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:49.888476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:49.899846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:49.901366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:49.902583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d3b3234b-34c3-45cc-955b-8c1479f2419a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160505 May 19 12:33:49.903100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1015/2151] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:50.876014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:50.876676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:50.883662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:50.884108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:50.884910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:50.885338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:50.923216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:50.923806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:50.933768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:50.934208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:50.934972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:50.935377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:51.026687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:51.035131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a2d9ff95-573c-47cb-b2db-744617fd6bc1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163478 May 19 12:33:51.035901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1137/2152] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:50 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:51.068079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:51.076670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:51.078215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:51.079381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-831d3973-7fa9-480b-8e08-f5a16824c4e3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162045 May 19 12:33:51.079922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1016/2153] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:52.056699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:52.057791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:52.068793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:52.069468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:52.070669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:52.071408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:52.097755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:52.098362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:52.104947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:52.105392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:52.106151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:52.106574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:52.110619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:52.220824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:52.229133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c77d0531-4c69-4337-84fb-5a1d7f8f2bc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178922 May 19 12:33:52.229703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1138/2154] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:52.253756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:52.262735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:52.264208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:52.265400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7ae51ad5-2115-4f82-b7dc-6f9b7a1a181c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.171582 May 19 12:33:52.265991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1017/2155] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:53.245757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:53.246390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:53.253623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:53.254059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:53.254798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:53.255213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:53.283530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:53.284152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:53.291275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:53.291708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:53.292482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:53.292944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:53.386914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:53.395106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-344b52b2-4343-44a7-a7d6-7aece8098196 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153301 May 19 12:33:53.395684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1139/2156] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:53 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:53.424333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:53.433057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:53.434535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:53.435701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-89931265-73d8-4e39-93f3-29f58768f357 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156173 May 19 12:33:53.436264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1018/2157] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:54.414632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:54.415253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:54.422322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:54.422767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:54.423517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:54.423947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:54.453911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:54.454480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:54.461477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:54.461922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:54.462686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:54.463116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:54.564381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:54.572820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-25406d66-b05d-4159-a9b3-863b8670632a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164479 May 19 12:33:54.573471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1140/2158] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:54 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:54.595819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:54.604779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:54.606275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:54.607441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0e5626a2-f60e-439c-8068-200f2856b490 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157454 May 19 12:33:54.608021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1019/2159] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:55.594402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:55.595386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:55.605989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:55.606726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:55.608042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:55.608731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:55.626382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:55.627043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:55.632353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:55.634714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:55.635126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:55.635954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:55.636379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:55.749197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:55.757352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18be8bc0-cd6a-4be6-8792-00298695351b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169594 May 19 12:33:55.758003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1141/2160] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:55.770487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:55.779255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:55.780803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:55.782029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9cc33ffd-67c0-454b-8e2b-205ab7923b2b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160137 May 19 12:33:55.782596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1020/2161] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:56.775962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:56.776606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:56.784629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:56.785186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:56.786062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:56.786565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:56.800389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:56.800979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:56.807959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:56.808399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:56.809209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:56.809641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:56.917317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:56.925428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8645eb17-4031-46cf-92bd-eee245fee590 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154074 May 19 12:33:56.925884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1142/2162] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:56 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:56.939754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:56.948330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:56.949859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:56.951030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b214563e-4c1e-4eaa-a4ad-2eae7d62f809 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154586 May 19 12:33:56.951596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1021/2163] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:57.946622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:57.947603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:57.951193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=52 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:57.960241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:57.961020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:57.961993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:57.962412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:57.969304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:57.969914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:57.976759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:57.977213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:57.977978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:57.978408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:58.097991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:58.106029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-66e795c8-b38e-4c0f-ae25-0f1841145412 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166058 May 19 12:33:58.106586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1143/2164] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:58.109666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:58.118319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:58.119808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:58.120995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d2dc7dd6-aca5-4a87-972d-585f903ca5a3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.155707 May 19 12:33:58.121545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1022/2165] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:33:59.125646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:59.126412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:59.135024 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:59.135504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:59.136257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:59.136684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:59.139515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:33:59.140112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:33:59.147078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:59.147518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:59.148267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:33:59.148678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:33:59.273481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:59.279274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:59.282633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1dcd12be-fb5b-4d30-a13e-ae262669dcca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162619 May 19 12:33:59.283217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1144/2166] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:33:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:33:59.287836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:59.289470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:33:59.290647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-75f3a2b2-7ced-42e1-afe9-e7cd671202d7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.154985 May 19 12:33:59.291199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1023/2167] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:33:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:00.303908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:00.305155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:00.310124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:00.310730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:00.312210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:00.312680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:00.313446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:00.313911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:00.317388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:00.317832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:00.318576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:00.318997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:00.444606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:00.447068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:00.453967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:00.455487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:00.455851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ab2c40bd-04fd-4375-a80b-9517e39d7cc8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158552 May 19 12:34:00.456458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1145/2168] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:00 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:00.456773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38272f96-fd17-49b8-8b11-1ba5c9a17b00 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.150500 May 19 12:34:00.457352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1024/2169] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:01.477728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:01.478330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:01.479842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:01.480455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:01.485343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:01.485765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:01.487563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:01.487988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:01.488703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:01.489181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:01.489943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:01.490367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:01.643600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:01.648546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:01.652089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:01.652482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:01.654030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:01.655461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a0c145a-a6c9-483a-86c7-622015c03644 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.180687 May 19 12:34:01.656324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1025/2170] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:01.662241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f792f103-8d2b-48bd-884f-b989f621a158 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.190732 May 19 12:34:01.662834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1146/2171] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:02.678324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:02.679318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:02.687588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:02.688694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:02.690675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:02.691430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:02.693230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:02.693456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:02.697875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:02.698365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:02.699135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:02.699573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:02.826194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:02.826707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:02.834588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-43cff19f-8bbf-4b20-bed5-14f3b3574460 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153656 May 19 12:34:02.835158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1147/2172] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:02.835677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:02.837542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:02.838805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cc5c4203-40dd-47d4-aca7-4c1223eafaab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166940 May 19 12:34:02.839601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1026/2173] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:03.852632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:03.853400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:03.858251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:03.858962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:03.863614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:03.864132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:03.865120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:03.865625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:03.866946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:03.867464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:03.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:03.868895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:04.003847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:04.004499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:04.012680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:04.012920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c527dd47-690f-4291-8231-e5717a5f0015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164774 May 19 12:34:04.013311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1148/2174] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:03 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:04.014231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:04.015384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f2c93f0f-af4a-4921-9c18-5d1ed2a42fbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161867 May 19 12:34:04.015901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1027/2175] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:05.034095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:05.035055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:05.037710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:05.038708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:05.044339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:05.044815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:05.045582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:05.045991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:05.046613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:05.047071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:05.047834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:05.048255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:05.176293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:05.176774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:05.184744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a455e569-8874-4466-aebd-1f0e15686d9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157143 May 19 12:34:05.185116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:05.187509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1149/2176] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:05.187657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:05.187861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-22fc42c8-b3d4-46c4-af2c-90101078ce73 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156769 May 19 12:34:05.188409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1028/2177] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:06.206082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:06.207241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:06.211271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:06.212308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:06.216164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:06.216647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:06.217466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:06.217896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:06.219061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:06.219508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:06.220447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:06.220915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:06.353800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:06.354050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:06.362228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-458c9cac-3815-4c72-a00b-d9e4e7e48fcb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162765 May 19 12:34:06.362799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:06.363107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1150/2178] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:06.364308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:06.365488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a72641d1-3dd1-476d-beb0-f14e9a01e6be tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161288 May 19 12:34:06.366031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1029/2179] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:07.384150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:07.385273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:07.386374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:07.386985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:07.391757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:07.392199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:07.392977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:07.393402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:07.393895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:07.394344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:07.395107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:07.395540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:07.525540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:07.526123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:07.533768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-762c42c4-f794-4d75-8182-f5b2180f2798 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156285 May 19 12:34:07.534323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1151/2180] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:07.534635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:07.536140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:07.537335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f2138de2-ae36-4737-aba5-a2f4640d0451 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156729 May 19 12:34:07.537860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1030/2181] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:08.553288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:08.553873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:08.555623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:08.556369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:08.560909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:08.561346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:08.562104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:08.562655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:08.563233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:08.563680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:08.564425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:08.564892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:08.693506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:08.699227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:08.701938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-044fee59-481a-4f01-b180-c327a5c6609d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153006 May 19 12:34:08.702761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1152/2182] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:08.707840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:08.709398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:08.710537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b40186f1-7293-477b-878e-e720d4031dc0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.159028 May 19 12:34:08.714414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1031/2183] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:09.722121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:09.722835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:09.729751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:09.730353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:09.731263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:09.731707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:09.732464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:09.732933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:09.737278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:09.737720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:09.738470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:09.738892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:09.867421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:09.873677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:09.875921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-68073441-f7a7-4582-b089-1728c772ef73 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159773 May 19 12:34:09.876501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1153/2184] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:09 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:09.882719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:09.884274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:09.885495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-32318126-b801-4c76-b11f-f68d42d0ef88 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160290 May 19 12:34:09.886041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1032/2185] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:10.896779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:10.898519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:10.907434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:10.908409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:10.910302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:10.911099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:10.912334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:10.913071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:10.918326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:10.918764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:10.919515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:10.919938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:11.046940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:11.049695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:11.053089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:11.055559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-da8ec484-101c-4b7a-88e5-8a21997006d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165222 May 19 12:34:11.056365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1154/2186] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:11.059030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:11.060530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:11.061740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e9b6c3a0-ea82-4f14-9bbc-c339a76d62da tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161062 May 19 12:34:11.062298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1033/2187] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:12.076965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:12.078034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:12.084270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:12.085205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:12.088807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:12.089250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:12.089988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:12.090409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:12.093302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:12.093999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:12.094777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:12.095211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:12.226761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:12.227398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:12.236180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:12.236372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3089e644-8ef1-483e-aed3-7a2a2b018c67 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165817 May 19 12:34:12.237000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1155/2188] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:12.237822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:12.239203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4d69fa0b-de59-4df9-8480-bb2a848bf776 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162655 May 19 12:34:12.240670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1034/2189] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:13.256285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:13.257114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:13.259749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:13.260948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:13.265090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:13.265527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:13.266275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:13.266699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:13.267843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:13.268291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:13.269100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:13.269539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:13.399213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:13.399394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:13.407685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4acf9689-fa79-445a-9c83-66a5e345f8c1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157105 May 19 12:34:13.408201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:13.408387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1156/2190] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:13 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:13.409776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:13.411001 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0977bb86-1894-4b01-a622-d7490366a028 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156370 May 19 12:34:13.411558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1035/2191] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:14.428865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:14.429792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:14.433269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:14.433884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:14.437767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:14.438210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:14.438965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:14.439378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:14.440478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:14.441031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:14.441777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:14.442222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:14.574654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:14.575117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:14.583009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11878f27-1640-44ca-9293-22ab50de2511 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160606 May 19 12:34:14.583583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1157/2192] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:14.583703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:14.585304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:14.586491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3ed29b4d-5cb6-4f10-8b81-aff7c4c4cd6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.159465 May 19 12:34:14.586999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1036/2193] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:15.604421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:15.605022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:15.606896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:15.607483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:15.611971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:15.612414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:15.613203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:15.613629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:15.614423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:15.614864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:15.615611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:15.616035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:15.754721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:15.754875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:15.763098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b6e8846a-1ff5-4e9a-9ad2-dffb004ace4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165221 May 19 12:34:15.763407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:15.763662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1158/2194] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:15.765083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:15.766262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b1e9a7bd-b750-442e-ad1a-1291cecc8fb7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.164304 May 19 12:34:15.766812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1037/2195] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:16.783740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:16.784596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:16.789733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:16.790727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:16.795646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:16.796435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:16.797892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:16.798597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:16.800879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:16.801617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:16.802851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:16.803567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:16.932443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:16.933976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:16.940921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-748cf5aa-ebd3-4fbf-a65c-e273f593cec1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163725 May 19 12:34:16.941499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1159/2196] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:16.942617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:16.944197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:16.945397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8430acdd-c915-490c-80ce-ac361c2e6718 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162167 May 19 12:34:16.945978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1038/2197] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:17.961477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:17.962465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:17.967433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:17.968470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:17.973596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:17.974325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:17.975558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:17.976258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:17.978559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:17.979372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:17.980133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:17.980560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:18.112761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:18.117944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:18.120989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f07a05b3-3e9b-4ac4-92d9-156a7f289fdc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166003 May 19 12:34:18.121932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1160/2198] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:18.126881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:18.128352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:18.129536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3df0ee12-3e26-431a-b68e-7863402113c4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.168516 May 19 12:34:18.130083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1039/2199] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:19.142383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:19.143623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:19.151700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:19.152716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:19.154551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:19.155371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:19.156708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:19.157423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:19.162651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:19.163332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:19.164578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:19.165303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:19.291141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:19.291309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:19.299569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5d7e2235-987b-40f2-98e6-d5cc51164527 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163645 May 19 12:34:19.300016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:19.300142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1161/2200] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:19.301547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:19.302718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2245bd1e-050f-4047-b3f0-8d9f40ffc738 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157705 May 19 12:34:19.303267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1040/2201] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:20.324532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:20.325602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:20.328266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:20.329398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:20.336673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:20.337412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:20.338758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:20.339183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:20.340304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:20.340787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:20.341550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:20.341948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:20.472425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:20.473432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:20.481173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-95d250f3-1fbf-4b67-9b7a-17fc2df08647 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163065 May 19 12:34:20.481741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1162/2202] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:20 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:20.482060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:20.483542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:20.484755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de80a2e2-f136-43a1-9734-d31d33b11b3d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.164179 May 19 12:34:20.485330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1041/2203] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:21.502858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:21.504072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:21.507453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:21.508279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:21.513359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:21.513804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:21.514937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:21.515220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:21.516251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:21.516861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:21.517824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:21.518265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:21.536399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:21.646549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:21.649836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:21.655314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:21.657096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:21.657852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c5ec231-4f8b-46f0-ad71-2c87ac7c26e2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161976 May 19 12:34:21.658403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c096cfea-dd60-4af4-9a26-aff214f31e27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157581 May 19 12:34:21.658564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1163/2204] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:21.658951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1042/2205] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:22.679280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:22.680401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:22.680607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:22.681432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:22.688844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:22.689169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:22.689333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:22.690172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:22.690454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:22.691528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:22.692306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:22.692756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:22.820846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:22.822533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:22.829026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11aaf8ff-2da9-4224-b18f-63ba31e9ff4d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156213 May 19 12:34:22.829634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1164/2206] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:22 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:22.831087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:22.832572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:22.833725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3bf119ec-24cc-44c7-b0b8-6669c6f359cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.159941 May 19 12:34:22.834408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1043/2207] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:23.848295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:23.849077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:23.852684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:23.853479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:23.857202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:23.857641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:23.858387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:23.858810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:23.861084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:23.861600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:23.862451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:23.862882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:23.992043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:23.993989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:24.001007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8651d52d-2972-4130-bb85-fbb2215055b2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157786 May 19 12:34:24.001586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1165/2208] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:23 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:24.003021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:24.004501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:24.005680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-76673fcf-46d0-4057-b1a4-1fe8df742e8a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157857 May 19 12:34:24.006192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1044/2209] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:25.022365 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:25.023329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:25.027053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:25.027940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:25.031939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:25.032381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:25.033169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:25.033591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:25.034869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:25.035323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:25.036089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:25.036522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:25.166123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:25.167697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:25.174358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-012e01c3-39bb-472d-be6d-fffb29f48ad5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158426 May 19 12:34:25.174936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1166/2210] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:25 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:25.176020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:25.177524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:25.178677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-25283941-987b-4cfc-b5d5-b609559df030 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.158149 May 19 12:34:25.179237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1045/2211] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:26.196460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:26.197679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:26.201342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:26.201956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:26.204291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:26.206738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:26.207237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:26.208084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:26.208605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:26.209874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:26.210317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:26.211071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:26.211500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:26.343717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:26.345424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:26.351944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e99c0cff-5a32-45af-b121-91363ba7c688 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162981 May 19 12:34:26.352510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1167/2212] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:26 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:26.354101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:26.355606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:26.356833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d96f095a-1c80-408d-b386-aefac7751157 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.161156 May 19 12:34:26.357386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1046/2213] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:27.373789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:27.374721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:27.378636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:27.380388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:27.385929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:27.386656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:27.387689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:27.388111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:27.389230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:27.389670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:27.390421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:27.390845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:27.520053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:27.520216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:27.528309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5c9b5c46-5549-402e-848b-a25bb12d64de tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161671 May 19 12:34:27.528958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1168/2214] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:27.529441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:27.530906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:27.532096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2e2fe746-993a-4db4-b849-f568b76d126b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.160162 May 19 12:34:27.532838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1047/2215] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:28.546063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:28.546696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:28.549033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=52 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:28.550241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:28.550858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:28.554525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:28.554966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:28.555738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:28.556194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:28.557722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:28.558167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:28.558961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:28.559391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:28.686171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:28.690243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:28.694815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:28.696293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:28.697480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f2231496-0267-441a-8aa8-da8b67126bc3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.151112 May 19 12:34:28.698169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1048/2216] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:28.698658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-157d0e84-4b76-422f-9101-293d2859f716 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156461 May 19 12:34:28.706803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1169/2217] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:28 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:29.718958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:29.719529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:29.721848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:29.722549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:29.728939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:29.729389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:29.730412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:29.730945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:29.731122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:29.731603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:29.732355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:29.732812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:29.863166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:29.865046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:29.871840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:29.873643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1406550e-3cd9-47e0-a1a5-19d97c183199 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161176 May 19 12:34:29.873804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:29.874181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1170/2218] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:29 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:34:29.877507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-20d467dd-03ff-4b2a-a821-ab2fc5c73a0d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.165366 May 19 12:34:29.878993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1049/2219] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:30.894884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:30.895862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:30.899875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:30.900915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:30.905692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:30.906131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:30.906889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:30.907306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:30.908463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:30.908941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:30.909694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:30.910124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:30.927892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:31.039200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:31.039374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:31.047570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e645d231-a702-4eff-9d41-0d78f5640480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159451 May 19 12:34:31.047927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:31.048441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1171/2220] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:30 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:31.049722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:31.050873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-12709caf-385d-43ac-bf48-ef930bd87b85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157449 May 19 12:34:31.051429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1050/2221] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:32.069342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:32.070862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:32.073699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:32.074802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:32.079228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:32.079656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:32.080404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:32.080856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:32.081766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:32.082219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:32.082965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:32.083400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:32.215915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:32.224374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d8b076fc-5254-42c9-b6f4-67041f7f4032 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161439 May 19 12:34:32.225022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1172/2222] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:32.443126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:32.452170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:32.453729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:32.454925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6a7707a4-9df3-4592-9884-4df3216bc247 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.387815 May 19 12:34:32.455732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1051/2223] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:33.241249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:33.241828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:33.249936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:33.250380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:33.251148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:33.251576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:33.388622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:33.397110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c1b6892-3bf5-4b45-8099-5d4cbc3e7385 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159859 May 19 12:34:33.397659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1173/2224] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:33 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:33.478377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:33.479860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:33.487284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:33.487720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:33.488462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:33.488915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:33.621462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:33.630673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:33.632241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:33.633419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ff97cef5-f11f-4618-b0b1-2a01f60aed1c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.162390 May 19 12:34:33.633966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1052/2225] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:34.418709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:34.419359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:34.427349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:34.427863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:34.428702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:34.429201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:34.579311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:34.587721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-64a3756f-44be-4d74-8332-ca7eede6380e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.175508 May 19 12:34:34.588291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1174/2226] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:34.652702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:34.653341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:34.659778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:34.660240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:34.661026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:34.661451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:34.794818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:34.803572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:34.805078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:34.806253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c74d28ec-6c81-40a6-a400-e7561eca3a27 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.158504 May 19 12:34:34.806811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1053/2227] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:35.609474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:35.610393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:35.621417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:35.622129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:35.623697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:35.624404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:35.766919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:35.775080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dea9cb15-30eb-42d8-a1cf-222d322418fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.172285 May 19 12:34:35.775650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1175/2228] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:35 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:35.825261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:35.825873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:35.832669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:35.833143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:35.833913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:35.834338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:35.979719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:35.988492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:35.990032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:35.991191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-04a73c72-826d-4d63-b66e-97cc958566a8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.169981 May 19 12:34:35.991784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1054/2229] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:36.795436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:36.796302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:36.807192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:36.807928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:36.809210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:36.809914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:36.954128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:36.962876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b95dfef7-f0e0-4836-bbfb-bb71f6cef421 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.174006 May 19 12:34:36.963455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1176/2230] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:37.013143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:37.013948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:37.022886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:37.023323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:37.024317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:37.025026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:37.163877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:37.172760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:37.174249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:37.175418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-91cc8752-d25a-469d-9cb6-9dddd0930337 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.168631 May 19 12:34:37.175981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1055/2231] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:37.989689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:37.990706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:37.999505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:38.000166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:38.001148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:38.001689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:38.154168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:38.162247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d553abda-48d0-4205-9a41-0b94a94a3ea6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.179140 May 19 12:34:38.162815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1177/2232] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:37 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:38.193843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:38.194400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:38.201243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:38.201687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:38.202462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:38.202885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:38.338020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:38.351650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:38.355449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:38.356663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-469a7eed-f97f-42bc-b410-07447302c5fc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.166841 May 19 12:34:38.357285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1056/2233] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:39.183355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:39.184302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:39.192018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:39.192487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:39.193315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:39.193769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:39.325600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:39.333842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ea0de00b-0588-4455-9400-a31d911f0480 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156945 May 19 12:34:39.334411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1178/2234] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:39.374754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:39.375356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:39.382484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:39.382924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:39.383673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:39.384074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:39.516370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:39.525691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:39.527181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:39.528351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-91dadf6d-7eaf-421c-b221-65142b2c045d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.157833 May 19 12:34:39.528941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1057/2235] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:40.351988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:40.352583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:40.359687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:40.360129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:40.360915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:40.361339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:40.501719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:40.510360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-42295fe9-e4c7-4bc1-ab1f-f47e4c85fd15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162232 May 19 12:34:40.511002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1179/2236] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:40.547068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:40.547732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:40.554558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:40.555014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:40.555770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:40.556216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:40.694760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:40.703391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:40.704916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:40.706110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34d3498a-2c20-46be-b04a-5bcb668e1e86 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.163222 May 19 12:34:40.706668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1058/2237] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:41.530920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:41.531708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:41.538100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:41.538539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:41.539280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:41.539708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:41.543692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:41.698056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:41.706567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-515bc383-4119-4bb6-b329-f62fd9137e59 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.181055 May 19 12:34:41.707167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1180/2238] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:41 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:41.724561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:41.725273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:41.732049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:41.732618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:41.733398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:41.733831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:41.865441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:41.874434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:41.875898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:41.877092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-18a0d4ac-71a0-46be-a14c-84f28049fa41 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1799 microversion: 2.20 time: 0.156449 May 19 12:34:41.877655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1059/2239] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:42.665833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-0ec9a2f4-91fd-47a7-bb31-ed1e06ac595d req-e0310d8e-8b2f-44ee-abee-b1dafd294ced service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b642ea31-4427-475b-8374-591f4dcbadc8", "name": "network-vif-deleted", "tag": "7b8465bd-5cd9-45e2-af0f-ea5a473e22bb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:34:42.674466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-0ec9a2f4-91fd-47a7-bb31-ed1e06ac595d req-e0310d8e-8b2f-44ee-abee-b1dafd294ced service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:42.674956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-0ec9a2f4-91fd-47a7-bb31-ed1e06ac595d req-e0310d8e-8b2f-44ee-abee-b1dafd294ced service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:42.686652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-0ec9a2f4-91fd-47a7-bb31-ed1e06ac595d req-e0310d8e-8b2f-44ee-abee-b1dafd294ced service nova] Creating event network-vif-deleted:7b8465bd-5cd9-45e2-af0f-ea5a473e22bb for instance b642ea31-4427-475b-8374-591f4dcbadc8 on hv-cinder-80403 May 19 12:34:42.691862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-0ec9a2f4-91fd-47a7-bb31-ed1e06ac595d req-e0310d8e-8b2f-44ee-abee-b1dafd294ced service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.158623 May 19 12:34:42.692857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1181/2240] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:34:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:34:42.724123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:42.724835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:42.731683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:42.731983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:42.732798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:42.733981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:42.866691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:42.875286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-187dd7db-6613-4355-b65a-1e46ac1e2d8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155702 May 19 12:34:42.875736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1060/2241] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:42 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:42.896114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:42.896836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:42.903382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:42.903827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:42.904630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:42.905092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:42.967818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:42.976398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:42.977929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:42.979061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d50310cf-4caf-4ca6-ac48-e24cbe3276a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087281 May 19 12:34:42.979597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1182/2242] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:43.893194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:43.893803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:43.900826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:43.901270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:43.902023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:43.902449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:43.997130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:43.997792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:44.005044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:44.005496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:44.006253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:44.006681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:44.035278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:44.043906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c69f75b2-c991-4df4-97df-44777b9c8a49 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154638 May 19 12:34:44.044489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1061/2243] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:43 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:44.079837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:44.092372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:44.094715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:44.096502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be738688-4df0-458e-94fe-eea585bc2c13 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102987 May 19 12:34:44.097311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1183/2244] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:45.065317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:45.066349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:45.077307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:45.077901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:45.079095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:45.079778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:45.118251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:45.118861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:45.126254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:45.126702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:45.127468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:45.127891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:45.191936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:45.200784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:45.202380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:45.203541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4fd257d7-5865-4f5f-bca8-8466af60d320 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089247 May 19 12:34:45.204116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1184/2245] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:45.214816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:45.222847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c2bda283-8471-43de-a3a2-92c46c143251 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164137 May 19 12:34:45.223401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1062/2246] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:45 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:46.223059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:46.223863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:46.230604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:46.231051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:46.231832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:46.232327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:46.239508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:46.240262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:46.247163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:46.247702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:46.249055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:46.249569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:46.320041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:46.329443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:46.330948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:46.332132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3bfb349f-a22f-49db-9568-c3b9957d53c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.113266 May 19 12:34:46.338574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1185/2247] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:46.385020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:46.393248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6cd12a36-e196-4847-983e-216012e31776 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157745 May 19 12:34:46.393787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1063/2248] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:47.354013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:47.354938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:47.366135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:47.366866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:47.368106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:47.368861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:47.412755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:47.413422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:47.422302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:47.422888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:47.423880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:47.424465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:47.444029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:47.452494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:47.454002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:47.455133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fc283279-4c23-4890-9366-195fd7af61dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107807 May 19 12:34:47.455713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1186/2249] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:47.559322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:47.567946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-938ce4d4-acd3-49b2-ad28-1dc550d1cd64 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161053 May 19 12:34:47.568525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1064/2250] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:47 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:48.473575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:48.474204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:48.481201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:48.481636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:48.482420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:48.482843 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:48.545941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:48.554813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:48.556303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:48.557552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d4556b7-4676-4909-85f5-2cf98611b423 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087986 May 19 12:34:48.558110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1187/2251] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:48.585395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:48.586002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:48.592496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:48.592970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:48.593717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:48.594143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:48.740116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:48.750865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15b75d68-128f-4494-8dc3-e24ffeb731b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169285 May 19 12:34:48.751467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1065/2252] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:48 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:49.580035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:49.581107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:49.592045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:49.592491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:49.593486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:49.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:49.669333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:49.682566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:49.684494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:49.685984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4c3f511f-4956-46fb-9ffb-eb0b8f3821d9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.112762 May 19 12:34:49.686537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1188/2253] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:49.771551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:49.772436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:49.782103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:49.782552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:49.783313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:49.783742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:49.912522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:49.920797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4d987382-7387-4a7c-aa74-67205cca2115 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155961 May 19 12:34:49.921527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1066/2254] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:50.706285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:50.706992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:50.715559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:50.716095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:50.717057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:50.717664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:50.782002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:50.790540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:50.792005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:50.793153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14122f2b-5e6d-4f95-af42-40e73a3214ac tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091933 May 19 12:34:50.793715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1189/2255] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:50.942332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:50.943306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:50.954721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:50.955455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:50.956706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:50.957421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:50.963897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:51.091447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:51.102371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-508b03e5-eb06-449e-bf38-b72fda58c9df tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166416 May 19 12:34:51.103116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1067/2256] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:50 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:51.814361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:51.815034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:51.823110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:51.823622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:51.824493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:51.825021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:51.893922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:51.902499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:51.903972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:51.905123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f6cf1c47-9551-49bf-a1d9-652be80e0d52 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096439 May 19 12:34:51.905760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1190/2257] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:52.124524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:52.125397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:52.141701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:52.142147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:52.142907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:52.143340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:52.287111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:52.295652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-98b78cb8-af72-4fc1-a608-386ac9bcee8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178396 May 19 12:34:52.296244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1068/2258] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:52.926478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:52.927375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:52.938376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:52.939100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:52.940322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:52.941101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:53.008114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:53.016666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:53.018145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:53.019265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a14e004-9f2b-45f7-9642-a83ef827a28c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099359 May 19 12:34:53.019828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1191/2259] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:53.314489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:53.315093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:53.322268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:53.322743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:53.323514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:53.323957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:53.454054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:53.462672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8b2da0f9-18e2-4778-8ab6-57cc90de943c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.152801 May 19 12:34:53.463176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1069/2260] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:53 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:54.041717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:54.042720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:54.053784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:54.054521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:54.055899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:54.056610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:54.123743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:54.132393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:54.133907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:54.135048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11f55948-e095-436f-9a2e-587db689dedd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100007 May 19 12:34:54.135615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1192/2261] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:54.484575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:54.485258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:54.493753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:54.494306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:54.495273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:54.495803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:54.643250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:54.653660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-87b3898c-b33c-4455-9416-a8c38fef10ec tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.175528 May 19 12:34:54.654235 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1070/2262] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:54 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:55.155854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:55.156487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:55.163233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:55.163682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:55.164417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:55.164880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:55.226664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:55.235730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:55.237249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:55.238377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a5cddb2-ff13-40d6-9361-a8b7ebb15074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086587 May 19 12:34:55.238929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1193/2263] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:55.671704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:55.672334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:55.679182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:55.679621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:55.680368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:55.680817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:55.815935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:55.824510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-148dc302-d15b-4d20-80cd-aea6b097eac1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156847 May 19 12:34:55.825151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1071/2264] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:56.257554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:56.258187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:56.264859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:56.265444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:56.266204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:56.266663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:56.328938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:56.337919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:56.339418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:56.340567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d37c02be-1a66-4129-ba2f-adbf1067cf5e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087130 May 19 12:34:56.341189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1194/2265] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:56.842181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:56.842836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:56.845300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:56.850988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:56.851435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:56.852197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:56.852717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:56.998541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:57.007078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e03161c2-cee8-4560-97b8-22217fa925ce tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168989 May 19 12:34:57.007657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1072/2266] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:56 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:57.362060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:57.362785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:57.370775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:57.371321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:57.372238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:57.372802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:57.439639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:57.448223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:57.449726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:57.450861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9587fb81-a112-41fc-b405-21dc64e699bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094507 May 19 12:34:57.451438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1195/2267] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:58.029011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:58.030200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:58.041904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:58.042620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:58.043844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:58.044545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:58.175863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:58.189498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-004ad696-7ffd-4fcc-be4b-0770780761a0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166708 May 19 12:34:58.190288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1073/2268] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:58.473199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:58.474056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:58.485064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:58.485789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:58.487008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:58.487705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:58.559167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:58.568014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:58.569537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:58.570717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-65ae0f26-e422-492e-a99d-22bdfe9843ce tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104189 May 19 12:34:58.571280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1196/2269] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:34:59.217577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:59.218399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:59.225735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:59.226204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:59.226997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:59.227447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:59.362422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:59.372995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8d2f35a4-76b3-4daf-a4ab-59600484ac6c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162300 May 19 12:34:59.373949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1074/2270] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:34:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:34:59.593127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:34:59.594182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:34:59.597998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=56 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:59.607101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:59.607827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:59.609066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:34:59.609495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:34:59.628005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:59.681100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:59.690303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:59.691832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:34:59.693073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f01cc329-be0a-4d1f-b23a-45b0993486dc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.106530 May 19 12:34:59.693603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1197/2271] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:34:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:00.394056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:00.394940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:00.404017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:00.404609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:00.405676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:00.406251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:00.557909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:00.566294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c00867be-343d-4961-a33e-584ff79a652d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178084 May 19 12:35:00.566856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1075/2272] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:00 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:00.715182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:00.716008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:00.723824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:00.724259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:00.725038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:00.725457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:00.790784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:00.799900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:00.801420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:00.802570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d762160d-7446-477c-9f83-3d4d2786356b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.093944 May 19 12:35:00.803137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1198/2273] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:01.588028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:01.589069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:01.597589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:01.598031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:01.598788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:01.599213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:01.737628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:01.746379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8c9c9db2-4ab3-47ab-b80d-a5138279e179 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164876 May 19 12:35:01.746926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1076/2274] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:01.824581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:01.825209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:01.832667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:01.833137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:01.833893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:01.834327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:01.905096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:01.913547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:01.915040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:01.916192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e37df272-375e-4b76-b5f2-10c8cbef85d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098002 May 19 12:35:01.916776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1199/2275] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:02.768178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:02.768861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:02.777338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:02.777840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:02.778610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:02.779057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:02.920729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:02.928892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-266fbe73-6e64-4efc-a8ae-9f571a60feea tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167859 May 19 12:35:02.929470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1077/2276] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:02.934221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:02.934826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:02.941877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:02.942318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:02.943077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:02.943505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:03.016359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:03.025590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:03.027023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:03.028176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1c3acf2a-94d4-4ca7-86b7-7c516e3f9947 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097850 May 19 12:35:03.028784 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1200/2277] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:03.950463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:03.951551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:03.962530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:03.963265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:03.964487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:03.965249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:04.046973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:04.047585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:04.055028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:04.055856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:04.057041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:04.057667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:04.113997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:04.126592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6659d288-0d8a-4a2a-b84e-01b0a24d1015 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.182275 May 19 12:35:04.127332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1078/2278] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:03 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:04.133219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:04.143279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:04.144790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:04.145931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ef7e5ba4-09ac-4855-9480-13cffb8d8f19 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.103202 May 19 12:35:04.146528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1201/2279] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:35:05.151625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:05.152613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:05.160771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:05.161216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:05.161978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:05.162402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:05.163404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:05.164037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:05.170677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:05.171119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:05.171880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:05.172309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:05.236878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:05.245433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:05.246913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:05.248052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-efe4a212-5249-4e2c-bfe3-e7e2929ea72f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.088661 May 19 12:35:05.249067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1202/2280] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:05.293440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:05.301532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fb6296c2-b24d-44c4-b941-66194a43a057 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156476 May 19 12:35:05.302137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1079/2281] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:06.266710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:06.267279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:06.274016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:06.274456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:06.275200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:06.275627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:06.323262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:06.323887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:06.330771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:06.331216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:06.331977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:06.332405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:06.343029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:06.351656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:06.353148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:06.354286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2da98a42-a35b-4942-a2a8-d89d7529397f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091919 May 19 12:35:06.354863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1203/2282] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:06.475385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:06.483609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ea29e555-4845-46a2-ac9b-3c5a57b8d947 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165423 May 19 12:35:06.484152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1080/2283] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:07.375476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:07.376121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:07.382746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:07.383181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:07.383925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:07.384346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:07.448293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:07.457398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:07.458882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:07.460029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f36326c2-f1ea-4ed0-8372-b42be6e2c6b3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090837 May 19 12:35:07.460590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1204/2284] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:07.502614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:07.503241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:07.510374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:07.510816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:07.511559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:07.511979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:07.652096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:07.661072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c3d53b25-feab-4273-8001-70b8e428a4fe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163395 May 19 12:35:07.661934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1081/2285] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:08.482936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:08.483925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:08.494917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:08.495653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:08.496938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:08.497626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:08.679328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:08.679893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:08.687190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:08.687637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:08.688410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:08.688874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:08.806554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:08.815336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:08.816854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:08.818007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7563f9ab-8820-4cc9-b0a9-35ca8ffe50b8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.341866 May 19 12:35:08.818570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1205/2286] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:08.826209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:08.834428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae7c3b46-c30a-4844-8fe4-4c7b10f467be tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159183 May 19 12:35:08.834962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1082/2287] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:09.840852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:09.841968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:09.849634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:09.850083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:09.850847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:09.851278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:09.851505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:09.852135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:09.858905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:09.859350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:09.860106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:09.860533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:09.914161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:09.923268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:09.924807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:09.925965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-722b31ca-27b4-4b76-b9bf-75df58e82477 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091708 May 19 12:35:09.926565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1206/2288] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:09.996262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:10.001161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:10.005902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5cfaec9b-ec8a-42b5-b592-a4b14cc3ebf0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158259 May 19 12:35:10.006474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1083/2289] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:09 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:10.947594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:10.948591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:10.957137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:10.957574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:10.958320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:10.958738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:11.020619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:11.023302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:11.023971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:11.029552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:11.030911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:11.031038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:11.031354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:11.032126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:11.032247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a975b83-98a7-4608-844f-20cf42a13de2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091261 May 19 12:35:11.032533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:11.033076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1207/2290] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:11.169555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:11.181819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3685cf94-d282-476d-9ca0-b9f942b3ac8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162541 May 19 12:35:11.182576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1084/2291] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:12.054838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:12.055910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:12.065503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:12.065941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:12.066679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:12.067099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:12.129282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:12.137753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:12.139214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:12.140389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b75f030f-b9e9-4cae-8774-8278057bc8cf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092484 May 19 12:35:12.141004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1208/2292] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:12.201097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:12.201808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:12.208560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:12.209035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:12.209774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:12.210239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:12.343755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:12.351741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-08d49e2f-15ae-4d9f-99d4-c1e4db90bc89 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154705 May 19 12:35:12.352275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1085/2293] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:13.162853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:13.164278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:13.173933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:13.174371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:13.175124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:13.175550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:13.243138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:13.252476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:13.254000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:13.255136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-08c65687-6f65-41a8-ba70-bf8011ef4872 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098926 May 19 12:35:13.255732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1209/2294] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:13.373474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:13.374612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:13.384629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:13.385094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:13.385841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:13.386263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:13.522495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:13.530713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2d9090ed-5984-4515-aa1f-ae8651875ef1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164137 May 19 12:35:13.531307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1086/2295] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:13 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:14.277765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:14.278638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:14.287006 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:14.287443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:14.288183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:14.288602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:14.529804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:14.538868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:14.540524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:14.541776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11fcdb79-86a9-4f14-9e33-015bf8491acf tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.270872 May 19 12:35:14.542499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1210/2296] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:14.547976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:14.548665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:14.555595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:14.556055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:14.556844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:14.557275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:14.691932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:14.700053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01b9bc9d-9341-442a-abbe-850edc05dcbe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156207 May 19 12:35:14.700625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1087/2297] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:15.364448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:35:15.365435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:35:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:35:15.562173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:15.563104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:15.572995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:15.573638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:15.574716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:15.575343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:15.646107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:15.656160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:15.658081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:15.659219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-939aada6-95b3-4b30-9314-a11823bdd6d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102768 May 19 12:35:15.659771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1211/2298] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:15.717685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:15.718291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:15.725178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:15.725624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:15.726376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:15.726806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:15.862250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:15.870588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b4d49f27-c7c1-406f-82b4-5186cba25574 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156826 May 19 12:35:15.871341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1088/2299] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:16.680767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:16.681780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:16.697761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:16.698637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:16.699860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:16.700578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:16.764927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:16.773802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:16.775256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:16.776372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d4bb4cb-8af9-4160-b8c7-c3dae32bbed0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102179 May 19 12:35:16.776949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1212/2300] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:16.891417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:16.892528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:16.899816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:16.900281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:16.901111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:16.901559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:17.057867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:17.066203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-65206f0d-2da5-4f60-b6f7-5368f3a2ab0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.181369 May 19 12:35:17.066791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1089/2301] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:17.797744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:17.798691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:17.806030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:17.806488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:17.807273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:17.807789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:17.877416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:17.886062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:17.887533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:17.888689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d6a63e51-6d44-4e66-9159-952d78add5d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097602 May 19 12:35:17.889252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1213/2302] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:18.087595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:18.088507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:18.098269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:18.098712 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:18.099460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:18.099860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:18.231971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:18.240174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b221fb6-7038-4523-b85f-28edaff7a8cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159153 May 19 12:35:18.240744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1090/2303] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:18 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:18.911193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:18.912125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:18.923377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:18.923991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:18.925330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:18.926040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:18.932696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:19.001677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:19.010240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:19.011715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:19.012883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9694d890-c251-4f7a-9002-74e509fb6c09 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.108479 May 19 12:35:19.013438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1214/2304] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:19.263678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:19.264589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:19.272788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:19.273234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:19.273983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:19.274410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:19.406408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:19.414555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-76d70e69-9753-4077-835f-579045787b33 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157545 May 19 12:35:19.415127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1091/2305] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:20.034309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:20.035253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:20.046343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:20.047078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:20.048328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:20.048984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:20.118171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:20.129208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:20.130613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:20.131760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9131d710-b3f1-4278-8687-d024dc010da9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104243 May 19 12:35:20.132296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1215/2306] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:20.435481 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:20.436306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:20.444510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:20.444987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:20.445744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:20.446203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:20.586206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:20.594362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-37f378c3-0a76-4224-93a4-c54af6457b80 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165486 May 19 12:35:20.594929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1092/2307] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:20 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:21.154383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:21.155317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:21.163439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:21.163853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:21.164602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:21.165059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:21.231210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:21.240476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:21.241675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:21.242807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2a1f7926-1f4a-4e5b-af9e-68c144bbb543 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095064 May 19 12:35:21.243337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1216/2308] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:21.615728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:21.616705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:21.627694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:21.628124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:21.628950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:21.629333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:21.770257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:21.778381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bf1fc94e-c3da-448b-b7a5-8ba55bb7cac9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169193 May 19 12:35:21.778926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1093/2309] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:22.262112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:22.262729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:22.269611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:22.270057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:22.270845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:22.271273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:22.339165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:22.348172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:22.349704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:22.350848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c92e0187-8177-4ac7-826d-81f75cd0505e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092719 May 19 12:35:22.351470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1217/2310] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:22.799723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:22.800362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:22.808395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:22.808965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:22.809850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:22.810357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:22.968065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:22.976572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a335e81d-55a8-43db-bd21-d89e212b3418 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.183599 May 19 12:35:22.977172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1094/2311] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:22 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:23.372303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:23.373658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:23.383952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:23.384470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:23.385602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:23.386156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:23.455290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:23.464114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:23.465645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:23.466756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-acdfff73-997f-4390-81c9-a4c3f67fdc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101093 May 19 12:35:23.467303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1218/2312] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:24.001366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:24.002218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:24.009154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:24.009596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:24.010348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:24.010807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:24.157035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:24.165664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-958b4991-6b76-4a11-adfb-13c964ed5d02 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171197 May 19 12:35:24.166257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1095/2313] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:23 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:24.488628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:24.489406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:24.499325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:24.500690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:24.501746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:24.503558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:24.595965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:24.604684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:24.606179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:24.607314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f4f539a4-acba-43d2-bd3a-690bef602bbd tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.124956 May 19 12:35:24.607874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1219/2314] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:25.187633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:25.188234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:25.195647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:25.196086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:25.196876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:25.197300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:25.340956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:25.351200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-16087f70-15ee-4f58-9d65-1f0c6ff75d8c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167513 May 19 12:35:25.351786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1096/2315] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:25 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:25.629285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:25.630017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:25.638834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:25.639249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:25.640215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:25.640812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:25.714054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:25.727240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:25.729569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:25.731257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2ec25487-2216-4def-8da0-b83d37db7aa6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107980 May 19 12:35:25.731992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1220/2316] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:25.852703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:35:25.853091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:35:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:35:26.368763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:26.369382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:26.376832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:26.377240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:26.377995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:26.378424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:26.523092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:26.531398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c1b98d65-80c8-4d09-a58e-0d7b85d4561c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166585 May 19 12:35:26.531971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1097/2317] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:26 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:26.755622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:26.756544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:26.767551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:26.767988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:26.768760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:26.769182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:26.832320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:26.841002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:26.842468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:26.843600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b54338d4-3ea9-47c2-9458-9924d02bf25f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094759 May 19 12:35:26.844146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1221/2318] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:27.557662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:27.558603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:27.562323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:27.568693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:27.569167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:27.569950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:27.570376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:27.706188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:27.714789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-221b6589-45a3-4bd6-86e9-30d4319c3692 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163761 May 19 12:35:27.715364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1098/2319] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:27.865691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:27.866726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:27.877764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:27.878207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:27.879088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:27.879401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:27.944813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:27.953321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:27.954885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:27.956053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-29f82dda-adf4-44e0-98a1-735f584d68f2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096955 May 19 12:35:27.956612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1222/2320] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:28.735620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:28.736508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:28.745168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:28.745777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:28.746859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:28.749358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:28.893437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:28.901982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-10f2c63c-b56e-4749-832b-7d6c26c952b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.172940 May 19 12:35:28.902559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1099/2321] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:28 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:28.978303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:28.979229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:28.988773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:28.989309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:28.990058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:28.990485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:29.060232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:29.071052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:29.073434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:29.075142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-42382b10-4b07-44c2-a47d-369680760e62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.103013 May 19 12:35:29.075886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1223/2322] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:29.921134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:29.921882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:29.930316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:29.930859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:29.931779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:29.932307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:29.951802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:30.069525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:30.082054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-36a2cbbc-c826-4e82-b760-f31adf3bed1d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165679 May 19 12:35:30.082816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1100/2323] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:29 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:30.216413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:30.217291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:30.218886 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:30.224428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:30.224892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:30.225653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:30.226071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:30.288901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:30.297415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:30.298922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:30.300060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6f04d35a-9012-4571-8376-44b4452cdce1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.203679 May 19 12:35:30.300627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1224/2324] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:31.103361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:31.104300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:31.113932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:31.114378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:31.115106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:31.115583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:31.252584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:31.261104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aaa4c8fa-d6bc-4788-aca3-a295de69fb53 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164315 May 19 12:35:31.261687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1101/2325] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:31 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:31.319883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:31.320528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:31.328563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:31.329107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:31.330031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:31.330529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:31.398427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:31.407300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:31.408869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:31.410007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cf65e90c-d9c0-44d2-af7f-a220fcaf8cf4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094771 May 19 12:35:31.410558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1225/2326] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:32.281940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:32.282666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:32.289911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:32.290346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:32.291087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:32.291507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:32.420551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:32.427763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:32.428307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:32.429108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ccc3acad-8cfb-4419-8843-f65183787304 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153520 May 19 12:35:32.429664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1102/2327] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:32.434657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:32.435095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:32.435854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:32.436273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:32.497760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:32.507189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:32.508684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:32.510020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-de6a6410-6f08-43b7-9f4c-1bef13dc8ad2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086098 May 19 12:35:32.510582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1226/2328] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:35:33.450250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:33.451164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:33.459497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:33.459964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:33.460745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:33.461161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:33.528116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:33.528744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:33.535402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:33.535833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:33.536582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:33.537048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:33.596167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:33.597818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:33.604249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2217c927-a687-4a2a-a453-a6031bc81c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160372 May 19 12:35:33.604846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1103/2329] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:33 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:33.606679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:33.608150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:33.609310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bfec86b3-e6e1-460e-b636-29f382f99c4d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.085121 May 19 12:35:33.609848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1227/2330] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:33.615200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:35:33.615356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:35:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:35:34.120424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:35:34.120815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:35:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:35:34.625038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:34.626318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:34.626866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:34.627463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:34.634655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:34.635101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:34.635857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:34.636262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:34.637472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:34.638004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:34.638797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:34.639298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:34.698066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:34.706986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:34.708469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:34.709645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4ae92372-9dba-472c-a0ad-ab1cbcf35fb4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087126 May 19 12:35:34.710203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1228/2331] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:34.778160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:34.786770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-203d9153-15f1-4c84-baf9-1305ccc5d992 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168634 May 19 12:35:34.787440 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1104/2332] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:35.727711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:35.728316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:35.735349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:35.735783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:35.736574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:35.737035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:35.803208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:35.804300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:35.805013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:35.811817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:35.812012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:35.812256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:35.813047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:35.813453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:35.813604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:35.814682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8864995c-e44d-45a3-aa42-7e0217c6eb5f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091038 May 19 12:35:35.815242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1229/2333] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:35.944735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:35.953159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e6593158-bc5c-49e5-bc2c-37f3ac7bc9c3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153013 May 19 12:35:35.953728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1105/2334] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:35 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:36.836960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:36.837861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:36.847654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:36.848088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:36.848870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:36.849292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:36.920212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:36.923550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:36.929567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:36.931026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:36.932149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-585b753f-4cd4-4464-8bea-67848af57116 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101738 May 19 12:35:36.932720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1230/2335] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:36.976853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:36.977569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:36.985224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:36.985670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:36.986547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:36.986980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:37.134556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:37.145698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5fc4ae4d-11e4-419c-89a0-ed7d4da99123 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.172858 May 19 12:35:37.146232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1106/2336] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:37.955473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:37.956092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:37.963704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:37.964163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:37.964974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:37.965411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:38.030296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:38.043518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:38.045885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:38.047565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4aedfdf1-0283-4ca1-b264-b34fb3391c2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098310 May 19 12:35:38.048359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1231/2337] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:38.166401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:38.167206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:38.178070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:38.178781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:38.180020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:38.180720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:38.315058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:38.323681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d42227a2-2879-4626-96c7-a2e9cd594714 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163889 May 19 12:35:38.324497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1107/2338] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:38 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:39.069773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:39.070690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:39.081774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:39.082501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:39.083717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:39.084418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:39.149813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:39.158706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:39.160172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:39.161340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86b9fabc-5f77-4e62-9371-1c45099b3ac3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098225 May 19 12:35:39.161890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1232/2339] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:39.344917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:39.345761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:39.356826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:39.357564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:39.358810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:39.359517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:39.497566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:39.506259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a7bcf6ed-03f6-4532-bfff-ef01a81510f7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167758 May 19 12:35:39.506735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1108/2340] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:40.183513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:40.184386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:40.194887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:40.195582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:40.196831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:40.197503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:40.265170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:40.274092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:40.275557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:40.276690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a555c5aa-eebb-42fd-8a1d-c8e0b225baf6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099787 May 19 12:35:40.277233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1233/2341] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:40.527662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:40.528730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:40.537315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:40.537756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:40.538526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:40.538951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:40.677817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:40.686206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4170bc53-9e7d-4e43-9a92-2f0a7cf12266 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164975 May 19 12:35:40.686772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1109/2342] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:41.298964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:41.299960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:41.310980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:41.311960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:41.313342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:41.315895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:41.383751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:41.392814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:41.394300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:41.395433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4cf1ae87-d444-4753-af2d-e2a1dd58d6c3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102998 May 19 12:35:41.395988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1234/2343] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:41.715719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:41.716597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:41.727351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:41.728091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:41.729392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:41.730106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:41.870273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:41.879209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5aaf56b6-531f-4ab7-a976-81b2270b11cc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170049 May 19 12:35:41.879789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1110/2344] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:41 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:42.422181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:42.423127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:42.430329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:42.430775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:42.431530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:42.431950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:42.502353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:42.513305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:42.515019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:42.516889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9fc8e967-5650-40aa-a2e3-e3d206e2e85d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101098 May 19 12:35:42.517535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1235/2345] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:42.900688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:42.901680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:42.911224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:42.911667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:42.912424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:42.912889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:43.054846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:43.063415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-98e8cd60-c6e4-4dce-b5f2-cc2a36150560 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169182 May 19 12:35:43.064008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1111/2346] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:42 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:43.536235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:43.536874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:43.543821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:43.544263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:43.545068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:43.545494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:43.612982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:43.622168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:43.623996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:43.625154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4ffab5bb-347a-42eb-8eb3-42fcbb6df5d1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.093294 May 19 12:35:43.625686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1236/2347] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:44.085319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:44.086346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:44.096676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:44.097127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:44.097874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:44.098292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:44.237798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:44.246561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-399b25e7-2690-4619-9075-9de8222d478f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168381 May 19 12:35:44.247136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1112/2348] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:44 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:44.653588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:44.654546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:44.665364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:44.666091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:44.667328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:44.668040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:44.744520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:44.754537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:44.756033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:44.757271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2379fad3-eb3b-46a7-b914-337413835c17 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.110317 May 19 12:35:44.757828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1237/2349] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:45.267981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:45.268528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:45.275562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:45.276009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:45.276796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:45.277222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:45.409275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:45.417544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b435e69a-1e1f-48c2-af59-045a3fd4b110 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155894 May 19 12:35:45.418122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1113/2350] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:45 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:45.779532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:45.780515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:45.790567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:45.791078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:45.791906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:45.792311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:45.853561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:45.864961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:45.867287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:45.869152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-048f678c-2808-465b-8cde-9142e043106e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095788 May 19 12:35:45.869957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1238/2351] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:46.440291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:46.440994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:46.448557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:46.449073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:46.449905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:46.450374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:46.585376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:46.593855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-88fd0b56-d18a-4bdb-8aac-7998b909af99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159635 May 19 12:35:46.594494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1114/2352] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:46.888487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:46.889249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:46.896607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:46.897081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:46.897828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:46.898256 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:46.968791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:46.977753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:46.979244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:46.980374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7f8e1ffa-25c6-4f23-92a5-f70fca054d4f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096023 May 19 12:35:46.980953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1239/2353] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:47.615572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:47.616529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:47.628117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:47.628996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:47.630051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:47.630754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:47.767964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:47.776522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8eb1da34-b1e2-49ac-a30b-2bb47b6e8974 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167522 May 19 12:35:47.777131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1115/2354] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:47 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:47.998984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:47.999545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:48.006449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:48.006892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:48.007639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:48.008061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:48.090882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:48.099795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:48.101310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:48.102481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-88a1471e-8704-42d9-9f7c-553a9d8a19df tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107703 May 19 12:35:48.103003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1240/2355] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:48.795066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:48.795636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:48.805289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:48.805851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:48.806915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:48.807498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:48.959624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:48.972108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-10c8e2b0-90c1-4dbe-9f5e-a9a1c306ae23 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.181611 May 19 12:35:48.972899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1116/2356] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:48 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:49.123611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:49.124542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:49.135547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:49.136275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:49.137575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:49.138278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:49.201867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:49.210662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:49.212131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:49.213291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9cb4d453-2c72-48a9-bab7-34434773aeab tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096466 May 19 12:35:49.213890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1241/2357] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:49.999008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:49.999977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:50.010339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:50.011073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:50.012310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:50.013083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:50.018851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:50.163097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:50.171305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-810d309f-573c-4b32-9dd5-faac25810198 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.179212 May 19 12:35:50.171895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1117/2358] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:50.235942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:50.236966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:50.248301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:50.249123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:50.250366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:50.251079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:50.326121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:50.335017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:50.336497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:50.337663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d2f92491-a931-40e6-a7c7-1e52119315c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.108294 May 19 12:35:50.338221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1242/2359] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:51.197692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:51.198658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:51.208033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:51.208478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:51.209276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:51.209703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:51.346021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:51.355720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c08b82dc-58bc-4cf0-85c5-0fff928dddfe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164549 May 19 12:35:51.355908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:51.356348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1118/2360] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:51 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:51.356506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:51.363861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:51.364307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:51.365099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:51.365545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:51.390767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:35:51.390965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:35:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:35:51.427351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:51.436181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:51.437707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:51.438837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3cfc9e44-1326-46d5-a88f-e4cd3bdcfc07 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086740 May 19 12:35:51.439440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1243/2361] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:52.374616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:52.375302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:52.382626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:52.383061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:52.383807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:52.384226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:52.457295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:52.457896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:52.464950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:52.465392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:52.466137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:52.466556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:52.522038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:52.530653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-659ade53-1e9b-4799-a70f-1e0f3c856089 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160587 May 19 12:35:52.530769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1119/2362] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:52.533067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:52.541842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:52.543333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:52.544489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e1f88de-0cbc-415d-9970-a090b1c105d5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091214 May 19 12:35:52.545082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1244/2363] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:35:53.553616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:53.554554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:53.564292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:53.564980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:53.565233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:53.565938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:53.566163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:53.566625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:53.575558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:53.576186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:53.577359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:53.578176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:53.640485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:53.649272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:53.650635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:53.651761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-41268a85-20e5-4f8f-b931-1300276d0074 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092239 May 19 12:35:53.652507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1245/2364] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:53.702040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:53.710638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-94d819ec-944c-4eca-8372-8f44b9888eac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166071 May 19 12:35:53.711490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1120/2365] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:53 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:54.674785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:54.675718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:54.686692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:54.687428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:54.688717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:54.689411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:54.727254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:54.728116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:54.735006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:54.735445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:54.736198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:54.736620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:54.751273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:54.760083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:54.761587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:54.762780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-179a6d42-5ade-4bf1-a4cc-0c34602ea870 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094656 May 19 12:35:54.763263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1246/2366] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:54.879137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:54.887195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-25497e19-92bb-4f7a-9061-ad59fd1dc700 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163792 May 19 12:35:54.887811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1121/2367] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:54 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:55.786989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:55.787845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:55.798797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:55.799355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:55.800260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:55.800723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:55.820345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:55.866542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:55.875724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:55.877248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:55.878432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-00a1321e-a1ff-46c9-a54d-f749146063d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100287 May 19 12:35:55.878987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1247/2368] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:55.905012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:55.905621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:55.912528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:55.913095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:55.913841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:55.914263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:56.046319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:56.054544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-235081b2-2f95-4903-b120-03cef0703f06 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153424 May 19 12:35:56.055107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1122/2369] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:56.900351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:56.901370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:56.910981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:56.911474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:56.912288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:56.912796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:56.981427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:56.994481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:56.996755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:56.998806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-762308ba-a247-4a20-af3d-8c8e0e393520 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104289 May 19 12:35:56.999564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1248/2370] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:57.075596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:57.076536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:57.087884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:57.088622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:57.089899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:57.090609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:57.227571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:57.235930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28131d91-1f97-4ed3-b9a5-a7521b4fb742 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166557 May 19 12:35:57.236502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1123/2371] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:58.022059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:58.023101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:58.034199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:58.034925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:58.036142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:58.036901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:58.106018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:58.114540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:58.115993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:58.117153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-66e1f249-3c25-4581-927c-43aa2f5b5970 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101823 May 19 12:35:58.117730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1249/2372] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:58.257786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:58.258626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:58.262345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:58.269941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:58.270385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:58.271143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:58.271567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:58.414155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:58.422280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d04acada-dfa4-42fe-8c7f-c8d393edd216 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171073 May 19 12:35:58.422844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1124/2373] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:35:59.140028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:59.141135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:59.152182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:59.152990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:59.154241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:59.155008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:59.219557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:59.228519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:59.231672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:59.231672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2c57a901-c15e-4ad6-87e4-c649d3a4036b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098302 May 19 12:35:59.231941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1250/2374] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:35:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:35:59.442545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:35:59.443144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:35:59.450935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:59.451428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:59.452382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:35:59.452914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:35:59.588153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:35:59.601058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eb8ec73b-29db-4683-af28-021f7fc80ac3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163798 May 19 12:35:59.601859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1125/2375] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:35:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:00.255690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:00.256293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:00.258890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:00.265345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:00.265878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:00.266720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:00.267309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:00.329086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:00.337943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:00.339414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:00.340549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3a039ad9-511f-4914-922d-510a23a74a94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089376 May 19 12:36:00.341140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1251/2376] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:00.625262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:00.626289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:00.634512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:00.635028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:00.635911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:00.636404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:00.772833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:00.780859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-991a52e3-8a0f-4238-9861-ad4086f42cde tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162391 May 19 12:36:00.781391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1126/2377] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:00 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:01.359851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:01.360544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:01.368705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:01.369255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:01.370136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:01.370636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:01.440498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:01.449396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:01.450881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:01.452023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac1ae468-c7c6-42d7-8979-8a81aecf6b61 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096787 May 19 12:36:01.452579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1252/2378] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:01.802387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:01.803209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:01.814272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:01.815003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:01.816244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:01.817006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:01.953781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:01.961959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-845cd2d5-6fe3-412d-9644-cfdd83f79772 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166327 May 19 12:36:01.962567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1127/2379] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:02.475310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:02.476250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:02.487318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:02.487968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:02.488899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:02.489328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:02.551632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:02.560298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:02.561794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:02.562924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5833ace0-90dc-461e-b98c-e32693be0514 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094647 May 19 12:36:02.563494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1253/2380] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:02.983547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:02.984515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:02.995499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:02.996229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:02.997525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:02.998646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:03.141111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:03.150884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d2e01422-af79-4801-8cce-363537ae7fae tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.174003 May 19 12:36:03.151473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1128/2381] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:03.585083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:03.586062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:03.596516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:03.597099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:03.597863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:03.598287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:03.672359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:03.681244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:03.682734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:03.683874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-925c56de-6d04-4950-9602-001050b4cdee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.105315 May 19 12:36:03.684432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1254/2382] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:04.170753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:04.171354 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:04.178885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:04.179301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:04.180588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:04.181052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:04.318950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:04.326987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9dfcd9c3-9ffa-4daa-a973-5580719bd8b8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162176 May 19 12:36:04.327551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1129/2383] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:04 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:04.705750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:04.706374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:04.713383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:04.713829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:04.714587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:04.715013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:04.780895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:04.789854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:04.791331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:04.792465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3cd01d31-dc43-44fc-8fba-0e71bfd078b5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092880 May 19 12:36:04.793051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1255/2384] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:05.351462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:05.352470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:05.364373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:05.365145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:05.366382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:05.367086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:05.517494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:05.525820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-79223377-3bbb-464a-a440-e1d32cf8663f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.180898 May 19 12:36:05.526389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1130/2385] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:05.814619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:05.815200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:05.821984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:05.822430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:05.823169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:05.823622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:05.886699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:05.895751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:05.897267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:05.898354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a8eb3058-e724-408c-b846-23a0e3b7248a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090246 May 19 12:36:05.898912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1256/2386] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:06.543332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:06.544017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:06.551117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:06.551566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:06.552315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:06.552785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:06.699823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:06.711448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-040ade27-4a9f-482b-8bd0-7ede54e42d15 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171690 May 19 12:36:06.712213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1131/2387] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:06.923123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:06.924267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:06.935317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:06.936050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:06.937319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:06.938019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:07.011669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:07.021570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:07.023057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:07.024194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-331b885c-842e-450d-a515-7a3c8153062d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107646 May 19 12:36:07.024769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1257/2388] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:07.735479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:07.736337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:07.743277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:07.743721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:07.744469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:07.744929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:07.895056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:07.903434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-56b9e90f-cc32-4051-adbe-519deea51799 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.174516 May 19 12:36:07.904023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1132/2389] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:08.046276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:08.047443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:08.054654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:08.055120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:08.055905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:08.056349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:08.120682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:08.129659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:08.131139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:08.132277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-621bac44-995e-4db3-80fe-4b323a12de94 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092591 May 19 12:36:08.132858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1258/2390] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:08.925139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:08.926061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:08.937258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:08.937986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:08.939247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:08.939677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:09.100922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:09.106464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:09.114546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-619c93e1-96fb-4abd-b7d2-eaeeb672b805 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.195733 May 19 12:36:09.115438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1133/2391] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:09.150170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:09.150807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:09.157766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:09.158199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:09.158942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:09.159361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:09.228569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:09.240250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:09.241780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:09.242926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e05ae47-4b3a-4104-a735-30ecf6fb3d62 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096617 May 19 12:36:09.243483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1259/2392] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:10.141051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:10.142165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:10.149204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:10.149641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:10.150496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:10.150865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:10.261311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:10.261914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:10.268901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:10.269366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:10.270122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:10.270546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:10.280611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:10.288622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-364b5aef-047d-4795-8ce0-79d7936565a9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154651 May 19 12:36:10.289212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1134/2393] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:10.334972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:10.344154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:10.345833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:10.346976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9b309bf6-e63f-479f-a31c-067a6073f89b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089572 May 19 12:36:10.347510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1260/2394] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:11.310397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:11.311233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:11.319247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:11.319688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:11.320435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:11.320889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:11.365712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:11.366529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:11.373954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:11.374390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:11.375154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:11.375583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:11.437470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:11.446773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:11.448352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:11.449513 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ec792a01-f059-48e4-9f59-0332e09a938f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087795 May 19 12:36:11.450071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1261/2395] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:11.460722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:11.468996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c617ed3c-7d54-491e-98f3-ce2d5213bbe8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165519 May 19 12:36:11.469557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1135/2396] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:12.467548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:12.468243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:12.475031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:12.475472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:12.476214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:12.476654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:12.486163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:12.486736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:12.493659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:12.494104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:12.494858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:12.495292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:12.541052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:12.549451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:12.550921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:12.552041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-360cb4c4-0e49-4126-9923-ac55b0484828 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.088498 May 19 12:36:12.552633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1262/2397] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:12.624063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:12.632245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de322035-2c30-48ae-b5fb-eb0c836ee506 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.149964 May 19 12:36:12.632820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1136/2398] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:13.573690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:13.574652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:13.585832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:13.586571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:13.587813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:13.588526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:13.653382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:13.655046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:13.655691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:13.662488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:13.662927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:13.663059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:13.663677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:13.664098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:13.664434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:13.665608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6fc9aa1a-9441-4bfd-b319-fc8130766df7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098479 May 19 12:36:13.666156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1263/2399] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:13.800549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:13.810298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-776223f0-ba27-4f3c-b64d-9ed938fd9c48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159098 May 19 12:36:13.810870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1137/2400] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:13 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:14.687825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:14.688848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:14.699820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:14.700567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:14.701864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:14.702577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:14.707631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:14.768883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:14.781960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:14.784421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:14.786336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8379a29f-f84d-43bf-a95f-dbb123b84398 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104779 May 19 12:36:14.787305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1264/2401] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:14.834350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:14.835356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:14.846251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:14.846704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:14.847468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:14.847900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:14.989983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:14.998822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0907eb81-21b8-4da2-acb9-7a4ba15bbf1c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170919 May 19 12:36:14.999402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1138/2402] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:15.812922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:15.814015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:15.821086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:15.821530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:15.822284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:15.822714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:15.883461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:15.892391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:15.893997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:15.895136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c23b58b-6bcc-45f8-9340-090c40df75f9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.088799 May 19 12:36:15.895691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1265/2403] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:16.019891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:16.020672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:16.027864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:16.028427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:16.029404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:16.030040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:16.166969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:16.177618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-93c052fe-8536-421f-9853-6b87652b65f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162796 May 19 12:36:16.178197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1139/2404] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:16.917563 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:16.918509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:16.927398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:16.927838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:16.928578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:16.929028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:17.000260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:17.009057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:17.010536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:17.011666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8de0fe3a-32cf-4273-9b36-24f4fdd1785a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100695 May 19 12:36:17.012216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1266/2405] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:17.195339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:17.195926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:17.202871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:17.203318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:17.204073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:17.204522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:17.337546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:17.345829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6f1f836e-fa9a-4d99-a2a5-6970f987566d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154613 May 19 12:36:17.346390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1140/2406] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:18.033653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:18.034424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:18.043471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:18.044062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:18.045105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:18.045672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:18.110708 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:18.119608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:18.121353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:18.122595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-222dfd5e-2921-4def-8251-27accbdb197d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095388 May 19 12:36:18.123758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1267/2407] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:18.366559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:18.367589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:18.377749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:18.378256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:18.379598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:18.380191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:18.514143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:18.522626 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-33048138-30c9-4e7f-a834-86d9f88a22ef tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162782 May 19 12:36:18.523185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1141/2408] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:18 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:19.154169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:19.154800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:19.164892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:19.165494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:19.166532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:19.167180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:19.234737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:19.243439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:19.244945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:19.246085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b1b9815b-b6d9-4182-beba-4f5e737ab90d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098261 May 19 12:36:19.246597 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1268/2409] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:19.539821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:19.540556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:19.547525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:19.547964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:19.548760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:19.549178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:19.684953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:19.693382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-29851cef-f7ed-4553-824f-7097e7491b40 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157537 May 19 12:36:19.693953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1142/2410] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:20.267643 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:20.268488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:20.279440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:20.280167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:20.281632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:20.282330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:20.345329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:20.354282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:20.355759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:20.356938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c0efc5d2-5e5b-4471-8012-f9632b5370d2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095916 May 19 12:36:20.357495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1269/2411] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:20.715013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:20.716010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:20.727331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:20.728069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:20.729370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:20.730079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:20.862890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:20.871176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15b571d4-9e3a-4724-9b54-2f7956192dc6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162769 May 19 12:36:20.871754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1143/2412] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:20 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:21.379396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:21.380317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:21.391347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:21.392188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:21.392968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:21.393410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:21.473142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:21.486265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:21.487744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:21.488910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6d8e716d-babe-4c5f-964e-4f4f49eeb687 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.116354 May 19 12:36:21.489461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1270/2413] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:21.892994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:21.894038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:21.907632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:21.908085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:21.908906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:21.909340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:22.046778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:22.055380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8182876b-596b-437f-a1a0-ca9297d4f6d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168803 May 19 12:36:22.055954 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1144/2414] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:22.512769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:22.512769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:22.523527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:22.524261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:22.525557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:22.526271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:22.592020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:22.605074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:22.607520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:22.609417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-05efa540-c518-4ce4-b588-487a34f15f1f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104094 May 19 12:36:22.610267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1271/2415] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:23.076906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:23.077832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:23.088808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:23.089539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:23.090776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:23.091550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:23.231396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:23.243022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-349c39b7-fc3c-4497-bf20-127d99c7fcf8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.172444 May 19 12:36:23.243784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1145/2416] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:23 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:23.637989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:23.638971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:23.646990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:23.647427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:23.648175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:23.648599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:23.713586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:23.724528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:23.726945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:23.728591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d49bc91-4076-496f-89b0-36c21e484a11 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096980 May 19 12:36:23.729310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1272/2417] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:24.266494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:24.267303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:24.274202 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:24.274637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:24.275385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:24.275804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:24.405536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:24.413717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1fa406c5-ae71-4b40-b887-3e73832d0d82 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153869 May 19 12:36:24.414293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1146/2418] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:24 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:24.752873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:24.753480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:24.760533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:24.760991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:24.761732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:24.762153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:24.824674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:24.833173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:24.834644 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:24.835775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d1b8b7e2-26f6-48d0-afa8-4f43ae6c0ec2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089111 May 19 12:36:24.836338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1273/2419] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:25.441824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:25.442713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:25.450609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:25.451051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:25.451945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:25.452358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:25.593608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:25.602621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f52874d8-5b1b-4e09-b782-2239ff7a9570 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167336 May 19 12:36:25.603197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1147/2420] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:25 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:25.858596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:25.859640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:25.867413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:25.867853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:25.868604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:25.869063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:25.937263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:25.945875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:25.947566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:25.948758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-303be4ad-5d99-4fd7-9ecb-18774d4c8b8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096750 May 19 12:36:25.949286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1274/2421] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:26.624715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:26.625744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:26.636890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:26.637638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:26.638861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:26.639833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:26.775827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:26.788540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b4a1b7a-6e1f-44cd-b880-c40fac8e2605 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170489 May 19 12:36:26.789405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1148/2422] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:26 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:26.970709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:26.971324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:26.978562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:26.979022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:26.979802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:26.980244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:27.051338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:27.059801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:27.061307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:27.062439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9e7c3815-a961-4f79-9b62-c430966961de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098218 May 19 12:36:27.062990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1275/2423] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:27.813619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:27.814465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:27.821686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:27.822179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:27.822967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:27.823441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:27.975037 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:27.986864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8610de72-e588-42a6-868f-24f2e01d9659 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.179758 May 19 12:36:27.987436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1149/2424] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:28.087292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:28.088198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:28.099082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:28.099757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:28.101046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:28.101746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:28.172549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:28.181221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:28.182693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:28.183818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d33001d6-cdf2-4a3f-89c3-69df3a1a07d8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.105476 May 19 12:36:28.184374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1276/2425] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:29.008614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:29.009564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:29.013292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=52 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:29.020392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:29.020869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:29.021618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:29.022046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:29.039926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:29.154732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:29.163008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0ba14795-2a6f-4d1c-81bc-c27954ea45d7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160882 May 19 12:36:29.163601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1150/2426] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:29 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:29.205882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:29.205882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:29.214107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:29.214586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:29.215459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:29.215919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:29.291871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:29.301013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:29.302504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:29.303689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ada404c6-bd39-4209-88bd-268d693c0118 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.104506 May 19 12:36:29.304254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1277/2427] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:30.184542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:30.185476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:30.193718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:30.194163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:30.194914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:30.195340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:30.325723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:30.326342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:30.329217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:30.334627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:30.335062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:30.335811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:30.336229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:30.339746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:30.348062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4faad782-8e34-4a02-8704-e5ab4e8f6ba5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170192 May 19 12:36:30.348668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1151/2428] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:30 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:30.397823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:30.406558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:30.408076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:30.409224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7f717d9c-224c-467c-8b34-ac37ddcbc4ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087671 May 19 12:36:30.409827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1278/2429] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:31.368675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:31.369599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:31.379278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:31.379710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:31.380457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:31.380898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:31.427285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:31.427977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:31.434733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:31.435263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:31.436069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:31.436562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:31.498798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:31.510013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:31.511741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:31.512940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e572046e-46f6-4e87-9d1e-204e8a0a016b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089518 May 19 12:36:31.513724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1279/2430] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:31.514516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:31.527305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f5d0f49-18d9-469c-8d97-02bc5430d244 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163713 May 19 12:36:31.527938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1152/2431] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:31 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:32.531476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:32.532086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:32.539309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:32.539754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:32.540504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:32.540975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:32.544687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:32.545310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:32.552178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:32.552618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:32.553393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:32.553813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:32.606501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:32.610015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:32.616414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:32.617927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:32.619073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e732892f-fcc0-4275-9133-3ba966a386e1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091593 May 19 12:36:32.619630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1280/2432] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:32.684936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:32.697086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f294353-48c7-4040-8876-748faea6cef5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155911 May 19 12:36:32.697812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1153/2433] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:33.641135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:33.642192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:33.653283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:33.654012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:33.655246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:33.656015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:33.717052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:33.717660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:33.720373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:33.724412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:33.724922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:33.725661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:33.726265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:33.735096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:33.736566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:33.737753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f15b77e-3a23-4990-b1fd-d00a4c8edc6f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.103194 May 19 12:36:33.738551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1281/2434] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:33.860995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:33.869506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ebd8609d-98de-4e1b-ac55-faa87e22211e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156448 May 19 12:36:33.870082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1154/2435] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:33 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:36:34.760701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:34.761613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:34.770820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:34.771267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:34.772018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:34.772449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:34.840836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:34.849431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:34.850911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:34.852049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-01d00dca-b1b6-4e48-8abc-1d5f8ce04c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098055 May 19 12:36:34.852623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1282/2436] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:34.887394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:34.887996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:34.894910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:34.895357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:34.896121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:34.896540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:35.041418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:35.054677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4e0bb9c3-f891-4b3f-b04f-98224685fbe3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170812 May 19 12:36:35.055431 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1155/2437] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:35.871935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:35.872922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:35.880290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:35.880849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:35.881599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:35.882023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:35.948224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:35.961475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:35.963533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:35.965312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c35f75f1-fe01-4db7-ba51-a91a2e1339c1 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097662 May 19 12:36:35.966045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1283/2438] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:36.072611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:36.073251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:36.080273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:36.080737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:36.081486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:36.081918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:36.226258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:36.234836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-65013b75-f96a-4beb-ba0b-8ffd4d46693a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166153 May 19 12:36:36.235414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1156/2439] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:36.987429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:36.988234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:36.998066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:36.998645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:36.999626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:37.000231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:37.075136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:37.083920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:37.085447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:37.086589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d14933fb-1050-4bd1-aca1-2c0e51ae70f6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.105416 May 19 12:36:37.087215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1284/2440] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:36 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:37.258802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:37.259360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:37.267382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:37.267828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:37.268584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:37.269050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:37.409052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:37.417204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-961d6050-0e5e-489f-8de6-23990c04b430 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163326 May 19 12:36:37.417780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1157/2441] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:37 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:38.109202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:38.110153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:38.120810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:38.121540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:38.123510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:38.124208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:38.186159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:38.197938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:38.200376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:38.202194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a08c4303-807a-446b-a757-aafa167edabe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099775 May 19 12:36:38.203077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1285/2442] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:38.438935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:38.439472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:38.446248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:38.446691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:38.447456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:38.447877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:38.580052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:38.588234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a08e1dd4-45f9-4f29-9f14-ba098e45324f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156093 May 19 12:36:38.588863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1158/2443] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:38 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:39.230462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:39.231384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:39.239967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:39.240407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:39.241193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:39.241620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:39.313523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:39.326606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:39.328946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:39.330339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6a0f28b7-90c8-4e9f-a306-d38af2c5bb2d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.106550 May 19 12:36:39.330894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1286/2444] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:39.609454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:39.610561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:39.621495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:39.622222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:39.623767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:39.624507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:39.764118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:39.772710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ed9ab7a-d370-48c9-9a1f-305b23bd492b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169620 May 19 12:36:39.773221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1159/2445] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:40.350925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:40.351620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:40.360209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:40.360841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:40.361754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:40.362291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:40.427831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:40.436458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:40.437994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:40.439189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e5919c8-2880-40e5-ad8f-74659c8ff674 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.093204 May 19 12:36:40.439747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1287/2446] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:40.790310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:40.790856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:40.797927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:40.798423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:40.799174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:40.799601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:40.942861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:40.950977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-323c3ca7-f9ad-4be2-b199-90bdf082646e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164718 May 19 12:36:40.951544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1160/2447] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:41.466413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:41.467121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:41.473582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.474083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.474659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.475274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.531101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:41.531597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:41.538129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.538927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.539932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.540589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.571892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:41.572056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:36:41.580443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:41.581949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:41.583074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9aa3d9e2-afcc-4378-930c-46108044a590 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.129369 May 19 12:36:41.583693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1288/2448] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:41.667479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6e12dcbb-30c6-4dd4-8e6a-9aa9d625eb28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 204 len: 0 microversion: 2.1 time: 0.139892 May 19 12:36:41.668169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1161/2449] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:36:41 2022] DELETE /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:36:41.676505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:41.677136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:41.683496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.683954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.684744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.685178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.820053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:41.832068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9f6f8a28-b065-4ebd-af64-0e1a83c2557d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159371 May 19 12:36:41.832997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1289/2450] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:36:41.968085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:41.968811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:41.975569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.976015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:41.976804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:41.977236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:42.116118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:42.125338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c290553d-2e9a-4f7e-925e-c66af78ab8ee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161453 May 19 12:36:42.125928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1162/2451] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:41 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:42.605538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:42.606582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:42.617993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:42.618734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:42.619987 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:42.620720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:42.685046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:42.697685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:42.700237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:42.702050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bc29dbbb-bdee-4cfe-9f24-f79b4ddc3319 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102854 May 19 12:36:42.702957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1290/2452] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:42.853739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:42.854741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:42.865132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:42.865878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:42.867114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:42.867824 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:43.002000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:43.010149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f3fd16cb-357b-40d5-b69f-5d99a5caaeab tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162849 May 19 12:36:43.010725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1163/2453] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:43.146962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:43.147717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:43.158142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:43.158873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:43.160112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:43.160898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:43.306094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:43.315452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-24ecc7ca-0f6e-41cb-b561-d1476d652a54 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.175024 May 19 12:36:43.316186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1291/2454] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:43 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:43.728090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:43.728740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:43.737549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:43.737989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:43.738738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:43.739166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:43.809531 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:43.818303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:43.819798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:43.820990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d6d30597-accd-461f-86bb-f3a5498885a9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098957 May 19 12:36:43.821559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1164/2455] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:44.031417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:44.032489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:44.043468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.044195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.045472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.046177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.192587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:44.205544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cdc4c4f3-ecbe-4a3c-a25e-111a927d2220 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.180698 May 19 12:36:44.206311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1292/2456] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:44.337337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:44.338370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:44.347790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.348164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.348938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.349361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.369400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:44.491191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:44.499552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3e09f8ae-01c9-44fe-8fe6-dfea7003fa6f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169338 May 19 12:36:44.500127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1165/2457] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:44 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:44.843820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:44.844938 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:44.851604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.852041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.852812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:44.853261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:44.920542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:44.929372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:44.930834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:44.931968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b17ab7c-c26f-465f-aef4-a21f96c98969 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094629 May 19 12:36:44.932552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1293/2458] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:45.227662 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:45.228690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:45.238315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.238799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:45.239557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.239989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:45.383040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:45.398368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-17b20726-ed0b-44bb-a3b3-ff6cb0b7a835 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.177508 May 19 12:36:45.399290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1166/2459] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:45.518755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:45.519360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:45.525954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.526394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:45.527124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.527544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:45.661087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:45.669123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5d3b33ed-3b37-495b-aa7f-7c1ea2e95809 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155705 May 19 12:36:45.669682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1294/2460] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:45 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:45.953350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:45.954135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:45.961027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.961468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:45.962222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:45.962647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:46.033472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:46.042176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:46.043666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:46.044883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-49999365-95e7-472a-93bd-6b78c6f482fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096448 May 19 12:36:46.045430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1167/2461] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:45 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:46.415447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:46.416083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:46.422462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:46.422934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:46.423686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:46.424109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:46.552325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:46.560570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac8bf46e-0f30-4559-8bcd-bc7b52c1f609 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.149024 May 19 12:36:46.561175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1295/2462] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:46.690301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:46.690918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:46.698981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:46.699503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:46.700383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:46.700942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:46.838834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:46.847033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-50114efb-f34f-43eb-942b-19ecf6c9e98f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162756 May 19 12:36:46.847579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1168/2463] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:47.065821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:47.066434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:47.072983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.073432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.074188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.074615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.142811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:47.151470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:47.152990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:47.154134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b4f52261-8c51-4506-afb0-cdaaade6d6a2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.093709 May 19 12:36:47.154695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1296/2464] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:47.581959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:47.582970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:47.593468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.594210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.595450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.596162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.742985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:47.755404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cbebb02b-4fd0-479b-a2f7-3e99888aed49 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.179720 May 19 12:36:47.756184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1169/2465] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:47.867021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:47.867637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:47.874985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.875435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.876245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:47.876737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:47.897777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:48.042459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:48.051896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-13ec60ca-fcea-4637-bbd1-33a6f226bd07 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.190921 May 19 12:36:48.052449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1297/2466] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:47 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:48.173196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:48.173702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:48.180056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:48.180500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:48.181303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:48.181735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:48.244575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:48.253607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:48.255070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:48.256391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a6879187-1c98-49dc-a299-69be75c51380 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087428 May 19 12:36:48.257083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1170/2467] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:48.776739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:48.777704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:48.785765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:48.786209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:48.786994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:48.787421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:48.928763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:48.940420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77d6a7aa-d06a-4896-a006-aeaaae67901d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170299 May 19 12:36:48.941002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1298/2468] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:49.073986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:49.074881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:49.082714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.083149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:49.083889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.084316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:49.274929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:49.275538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:49.282383 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.282829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:49.283591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.284013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:49.350610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:49.360051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:49.361615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:49.362751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-576587cf-3dce-4f27-aa43-b870dd2c3ead tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091842 May 19 12:36:49.363280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1299/2469] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:49.472165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:49.480327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-45f8bebb-8a1f-4dad-891e-7469ddea82f5 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.413192 May 19 12:36:49.480934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1171/2470] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:49.959042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:49.959682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:49.966918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.967407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:49.968202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:49.968670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:50.104373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:50.112960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6d7197e5-9587-4512-ab44-c471c47d0d1e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.158010 May 19 12:36:50.113532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1300/2471] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:50.387788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:50.387788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:50.399705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:50.401130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:50.403408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:50.404703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:50.470957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:50.484120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:50.486283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:50.487430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1bb4265f-b9e5-4308-ac59-3fcc001cf696 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107983 May 19 12:36:50.487994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1172/2472] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:50.497844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:50.498743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:50.505170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:50.505610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:50.506364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:50.506785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:50.639999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:50.648088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e8246bab-0f39-4d3d-8900-b89a921fc669 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154365 May 19 12:36:50.648666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1301/2473] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:50 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:51.134057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:51.135049 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:51.145478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.146203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.147450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.148158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.284963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:51.297050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-98452ec4-ca9d-4db1-a8d8-9592ec8f27bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169028 May 19 12:36:51.297867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1173/2474] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:51.506769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:51.507348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:51.515446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.515922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.516946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.517376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.583200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:51.595512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:51.596971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:51.598125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a36eafa4-338a-4c4c-b07f-7821db5b5800 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095767 May 19 12:36:51.598686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1302/2475] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:51.668455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:51.669517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:51.680376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.681210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.682455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:51.683166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:51.829685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:51.837765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a7c9281-dbd6-4974-b8a7-ffc679279c0a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.175971 May 19 12:36:51.838329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1174/2476] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:51 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:52.317007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:52.317674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:52.324770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.325259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:52.326108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.326554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:52.462165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:52.472769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-505163ea-06b5-405c-90d2-cfa7f2831f64 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160846 May 19 12:36:52.473346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1303/2477] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:52.625177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:52.626112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:52.632515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.632985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:52.633729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.634149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:52.694134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:52.702906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:52.704457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:52.705578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-546c77c3-e0fe-4a1d-9bc7-9e033ae6f021 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087545 May 19 12:36:52.706148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1175/2478] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:52.859519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:52.860611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:52.867911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.868357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:52.869150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:52.869578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:53.005107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:53.013138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-75fa0164-9b79-4c7f-aefc-344b8bdb3990 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160326 May 19 12:36:53.013679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1304/2479] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:53.497131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:53.498184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:53.505794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:53.506290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:53.507180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:53.507828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:53.660162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:53.668508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-febaaa40-4abe-494b-82b7-a9738697effd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.176911 May 19 12:36:53.669117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1176/2480] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:53.724506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:53.725299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:53.734403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:53.734986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:53.735947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:53.736375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:53.809043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:53.817823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:53.819313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:53.820449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-15d2310a-0494-4c71-8da5-e876f0c46c59 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100717 May 19 12:36:53.821035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1305/2481] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:54.034132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:54.034896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:54.045831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.046378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.047368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.047943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.215309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:54.227560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-91edf485-5347-45d8-b528-fecdf61efa32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.199063 May 19 12:36:54.228296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1177/2482] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:54 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:54.692390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:54.693076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:54.701158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.701670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.702532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.703029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.835554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:54.839021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:54.839709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:54.844356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cc2d94d7-d36a-46ec-b612-d91f52435d41 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.157025 May 19 12:36:54.845283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1306/2483] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:54.847083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.847538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.848313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:54.848764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:54.911915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:54.925324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:54.927612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:54.929450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-44ec46c7-ff4e-4e59-9ef2-3384b1eef7eb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094346 May 19 12:36:54.930299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1178/2484] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:55.251954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:55.252996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:55.263192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.263879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:55.265203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.265906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:55.402720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:55.411680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cef5242c-e264-43e6-86f4-c17f4c49ff84 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166720 May 19 12:36:55.412174 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1307/2485] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:55.864337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:55.864997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:55.872308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.873176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:55.873951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.874393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:55.949997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:55.950560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:55.957616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.958062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:55.958822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:55.959215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:56.008337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:56.016546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-528866f3-9cd8-43dd-bae7-f525ef07e4ce tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.157035 May 19 12:36:56.017147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1179/2486] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:56.021756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:56.030279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:56.031760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:56.032943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c123a25f-1849-44e6-954a-5b9375c10038 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086944 May 19 12:36:56.033490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1308/2487] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:55 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:56.432361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:56.433451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:56.441898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:56.442339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:56.443093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:56.443519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:56.578725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:56.586888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ab5bdeb-b947-4df6-b77b-ba9954347e62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161075 May 19 12:36:56.587459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1180/2488] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:56 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:57.037798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:57.038822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:57.049849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.050543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.051783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.052549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.054234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:57.055228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:57.065296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.065745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.066500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.066931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.128562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:57.137163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:57.138765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:57.139934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-459b8d31-eb8f-4550-920d-2f932a0852d6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092249 May 19 12:36:57.140491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1181/2489] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:57.184808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:57.192934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-69b8ce97-bef1-418c-a568-fcdf21a80612 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161819 May 19 12:36:57.193521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1309/2490] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:57.607465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:57.608489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:57.618692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.619413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.620630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:57.621389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:57.627675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:57.767555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:57.780102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-39159ccc-5020-426d-a611-6fad8e638ef7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178928 May 19 12:36:57.780944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1182/2491] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:58.160273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:58.161030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:58.169681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.171135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.172701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.173202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.209839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:58.210437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:58.217756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.218199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.218955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.219376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.240755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:58.249622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:58.251159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:58.252287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1ca1e307-cacb-44fa-9b05-617ab0df03ff tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097954 May 19 12:36:58.252897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1310/2492] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:58.348100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:58.356523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-73e84034-5496-4215-bf9b-ed2a4052e2c8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.150341 May 19 12:36:58.357124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1183/2493] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:58.801398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:58.802456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:58.812794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.813606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.814824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:58.815526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:58.950874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:58.959453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7ff2e3bd-1e63-4ce4-840c-26c58e6380e3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164798 May 19 12:36:58.960030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1311/2494] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:59.274466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:59.275424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:59.278977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:59.285906 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.286366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:59.287109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.287533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:59.354697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:59.367604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:59.369112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:59.370284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5e7ca7a8-34be-49f2-af17-1874a1443553 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102349 May 19 12:36:59.370833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1184/2495] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:36:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:36:59.374032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:59.374632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:59.381518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.381959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:59.382752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.383184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:59.516878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:36:59.525260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5c6e9820-2395-4fc8-b84e-b9ed763d9fd8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155262 May 19 12:36:59.525769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1312/2496] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:36:59.979800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:36:59.980838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:36:59.991272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.991996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:36:59.993260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:36:59.994021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:00.132333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:00.140898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cc431dff-1461-4196-9555-ff604b4ff4dd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167504 May 19 12:37:00.141473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1185/2497] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:36:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:00.392162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:00.392949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:00.395500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=70 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:00.402200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:00.402711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:00.403580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:00.404072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:00.470507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:00.479098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:00.480564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:00.481707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e98ee7a-0d76-41e3-87fb-10df6a74a265 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095887 May 19 12:37:00.482276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1313/2498] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:00.541493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:00.542086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:00.548996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:00.549438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:00.550183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:00.550605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:00.691250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:00.702704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-422ddaab-8727-4a9c-b66e-45557b5eb600 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164868 May 19 12:37:00.703441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1186/2499] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:01.162403 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:01.163392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:01.173535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.173972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.174719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.175140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.179290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:01.304139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:01.312746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a743a72-514b-448c-a80e-e6034f455de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156944 May 19 12:37:01.313317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1314/2500] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:01.503177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:01.503733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:01.510319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.510762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.511499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.511946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.600127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:01.613026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:01.615334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:01.617141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c2fd601b-d96d-42a4-b1c4-ad8e038f570a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.119774 May 19 12:37:01.617943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1187/2501] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:01.728887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:01.729831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:01.739119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.739561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.740313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:01.740789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:01.875653 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:01.887840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3b71f8a-bf8d-4d92-8739-47cb5ad46e85 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165174 May 19 12:37:01.888736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1315/2502] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:02.333312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:02.334307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:02.344680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.345297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:02.346080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.346526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:02.475564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:02.483633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-45879de1-7a74-45ac-9f5a-c9ff1d7be649 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156661 May 19 12:37:02.484201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1188/2503] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:02.642508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:02.644892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:02.649896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.650354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:02.651141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.651600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:02.717686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:02.726299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:02.727775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:02.728945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ef94b81d-a79e-4134-9acf-28ca2121d120 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092952 May 19 12:37:02.729570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1316/2504] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:02.909197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:02.909756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:02.916661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.917121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:02.917871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:02.918294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:03.062003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:03.071087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-41851da7-76ca-4dbd-97dc-72ae2561a7c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167515 May 19 12:37:03.071656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1189/2505] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:03.505210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:03.506194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:03.517282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:03.517714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:03.518460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:03.518881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:03.662466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:03.675641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-314a4a42-c95b-4129-9e91-fddc4a070fd6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176555 May 19 12:37:03.676499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1317/2506] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:03 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:03.750790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:03.751733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:03.758306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:03.758704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:03.759425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:03.759860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:03.825530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:03.834201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:03.835684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:03.836853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-93d620c8-db86-4eab-bc2f-9360ea2b7354 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092011 May 19 12:37:03.837634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1190/2507] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:04.088351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:04.088980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:04.095469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.095908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.096677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.097132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.244222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:04.252898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aceee730-8e87-4767-bce3-bf75afa7ca03 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168412 May 19 12:37:04.253474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1318/2508] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:04.701353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:04.702339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:04.710011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.710447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.711193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.711611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.853821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:04.855481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:04.856344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:04.862687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8cfa421b-ac20-449f-a294-87cbce897efe tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167804 May 19 12:37:04.863242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.863415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1191/2509] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:04 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:04.863688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.864473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:04.864936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:04.930039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:04.939020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:04.940556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:04.941728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5cd39a40-6eb7-42e7-899b-604572d132f7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090526 May 19 12:37:04.942294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1319/2510] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:04 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:05.274781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:05.275764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:05.284743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.285173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:05.285914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.286337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:05.424123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:05.432393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1518b39a-54c6-4393-9d70-1ba16a1a5706 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164015 May 19 12:37:05.432996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1192/2511] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:05.883826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:05.884896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:05.901492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.902239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:05.903503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.904224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:05.959797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:05.960447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:05.967290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.967726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:05.968490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:05.968951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:06.035947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:06.042772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:06.045939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:06.047434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:06.048616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-02e05c9e-6721-4a23-897d-793b8db1d01b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092932 May 19 12:37:06.049197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1193/2512] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:37:06.052520 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5edb2008-0045-460e-9389-620a39f76f2e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.175188 May 19 12:37:06.053511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1320/2513] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:06.453673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:06.454557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:06.461346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:06.461806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:06.462582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:06.463028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:06.598478 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:06.610852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b2428b5-2e16-45bf-851e-7a482085dc15 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163435 May 19 12:37:06.611659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1194/2514] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:07.070676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:07.071599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:07.080927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:07.081421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.081542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:07.081839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.082583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.083002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.088465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.088950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.089701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.090129 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.145345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:07.154048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:07.155549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:07.156692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c0f555de-3b79-47f5-a4a3-1d8209c20767 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092612 May 19 12:37:07.157706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1321/2515] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:07.221684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:07.230079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d10b13e3-146b-42cf-9b73-194228f49f47 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154737 May 19 12:37:07.230648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1195/2516] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:07.641664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:07.642648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:07.654417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.655474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.656833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:07.657659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:07.799121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:07.807402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eb37aeb9-86f3-42ae-8ed5-c55c03c20f0e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173377 May 19 12:37:07.807979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1322/2517] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:08.179025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:08.180018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:08.190045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.190493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.191245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.191671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.248604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:08.249250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:08.255891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.256409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.257276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.257775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.257916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:08.266615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:08.268094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:08.269281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42612e68-0e09-49f2-b42b-7f9d6b697c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096835 May 19 12:37:08.270043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1196/2518] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:08.393182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:08.404791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-67d4320b-86bb-4239-94e1-d0d3315e245e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160247 May 19 12:37:08.405561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1323/2519] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:08.828356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:08.829403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:08.839559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.840292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.841576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:08.842295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:08.980518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:08.988560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b5321698-2fc0-4acc-85c7-1505a4131c94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166584 May 19 12:37:08.989153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1197/2520] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:09.287452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:09.288085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:09.295638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:09.296107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:09.296928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:09.297370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:09.358752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:09.367729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:09.369273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:09.370412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d8cb8f23-9022-4fdc-ac66-8845772f4123 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087214 May 19 12:37:09.370969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1324/2521] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:09.422036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:09.422869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:09.429748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:09.430184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:09.430927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:09.431359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:09.567820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:09.571242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:09.576558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a0a1d910-c2a6-4663-9346-216999c085fb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158495 May 19 12:37:09.577159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1198/2522] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:09 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:10.014337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:10.015321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:10.022728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.023167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.023907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.024327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.167195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:10.175796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-747495f0-de75-4e4d-9e47-52893b296814 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167829 May 19 12:37:10.176406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1325/2523] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:10.393053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:10.394047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:10.404262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.405107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.406355 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.407062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.483974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:10.496579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:10.498465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:10.499582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19972163-1a59-4f72-aa7f-d4349cee2ccc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.113169 May 19 12:37:10.500142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1199/2524] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:10.597845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:10.598888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:10.609926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.610663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.611900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:10.612613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:10.752354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:10.761061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eb9a4f7f-7ed5-43e0-819d-77e5b3a564a8 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169806 May 19 12:37:10.761638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1326/2525] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:11.196499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:11.197778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:11.208778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.209613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.210908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.211630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.347717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:11.356263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a262746-9e35-4839-8f81-bfccc6b0b88b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166124 May 19 12:37:11.356874 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1200/2526] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:11.520027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:11.520626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:11.527253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.527657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.528406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.528916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.593788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:11.602463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:11.603895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:11.605053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-67f208fb-ba7a-4b84-a5a0-95c74371a0ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089833 May 19 12:37:11.605609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1327/2527] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:11.783423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:11.784268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:11.795366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.796099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.797409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:11.798139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:11.932676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:11.943377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cf28ba93-04ea-4e36-aecc-5cd7fd89e045 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166262 May 19 12:37:11.944169 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1201/2528] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:12.376348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:12.377298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:12.387566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.388237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:12.389426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.390068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:12.528819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:12.540802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-196a7893-e917-49f2-8a29-412bb71157e5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170760 May 19 12:37:12.541571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1328/2529] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:12.627069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:12.627767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:12.635155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.635668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:12.636537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.637070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:12.705053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:12.713698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:12.715144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:12.716310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-da5c53a1-f849-4313-b642-20a9863c3957 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095915 May 19 12:37:12.716891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1202/2530] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:12.964868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:12.965841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:12.976240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.977035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:12.978278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:12.978984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:13.123747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:13.135814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-011739ff-cdb9-465d-a0f2-28a95834a139 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.177077 May 19 12:37:13.136699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1329/2531] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:13.560621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:13.561254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:13.568229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:13.568667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:13.569447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:13.569876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:13.724625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:13.733032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ed2f3cc-e1db-470d-b65c-918debfd89c1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.176271 May 19 12:37:13.733581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1203/2532] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:13.734279 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:13.734872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:13.741837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:13.742278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:13.743027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:13.743452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:13.804971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:13.808340 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:13.814725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:13.816246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:13.817353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-57d8005a-ed90-480d-b703-12305a3095f8 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087173 May 19 12:37:13.817914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1330/2533] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:37:14.159058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:14.159709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:14.167739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.168288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.169252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.169900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.304308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:14.312616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa95b469-ee7a-4dbb-b15b-e749534f9341 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159833 May 19 12:37:14.313698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1204/2534] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:14.754587 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:14.755544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:14.766610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.767053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.767807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.768235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.835418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:14.836020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:14.842879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.843319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.844069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:14.844491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:14.909907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:14.910569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:14.918362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:14.919018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1819e48c-be50-4d5b-8d6c-e9d7ce660992 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171584 May 19 12:37:14.919610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1331/2535] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:14.919850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:14.921060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ef4ba2fe-765f-486c-bc9b-ec6381c0e264 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089617 May 19 12:37:14.921631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1205/2536] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:14 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:37:15.334345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:15.335337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:15.345631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.346374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:15.347612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.348320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:15.490899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:15.499320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-67e0271a-e340-4526-8447-0b23f062b9bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171488 May 19 12:37:15.500104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1332/2537] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:15.937495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:15.938067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:15.940063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:15.940677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:15.944738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.945184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:15.945965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.946380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:15.947524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.947969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:15.948863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:15.949155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:16.018130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:16.027072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:16.028579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:16.029747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1d17b3dc-c080-42f1-a730-17cb0bdef1c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.093591 May 19 12:37:16.030305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1333/2538] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:16.078083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:16.087177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0669d12f-04e0-4293-92e3-4bbbb515f69f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154151 May 19 12:37:16.087747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1206/2539] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:16.520283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:16.521123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:16.530642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:16.531244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:16.532437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:16.532888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:16.908608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:16.916839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-858ff3c6-8cc1-4f76-88a9-90adb2e0ad48 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.402824 May 19 12:37:16.917410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1334/2540] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:16 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:17.051515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:17.052872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:17.064008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.064455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:17.065315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.065745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:17.108986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:17.109544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:17.116592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.117063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:17.117813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.118230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:17.130453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:17.139076 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:17.140536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:17.141737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7f817519-a529-4a5f-9ba1-ca45f2b5d194 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096915 May 19 12:37:17.142299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1207/2541] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:17.255974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:17.264245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-decacfcd-1692-4f04-ac78-f825277df7f4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159200 May 19 12:37:17.264853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1335/2542] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:17.938140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:17.939071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:17.946519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.947217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:17.947909 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:17.948415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:18.085267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:18.097002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-71e03c60-63a5-430d-8e8a-89f3a882acee tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165202 May 19 12:37:18.097580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1208/2543] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:18.160289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:18.161081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:18.169991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:18.170567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:18.171598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:18.172163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:18.240461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:18.253706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:18.255935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:18.257688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7948fa20-e233-4c12-97bb-373e5bb2db53 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101878 May 19 12:37:18.258404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1336/2544] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:18.281953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:18.282558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:18.289190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:18.289632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:18.290361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:18.290792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:18.424630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:18.432979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-53042a92-5581-4016-be25-71e6f742ec9b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155009 May 19 12:37:18.433560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1209/2545] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:19.116430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:19.117227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:19.126626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.127271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.128370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.128996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.267744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:19.276039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0046d10b-926c-426e-b4d3-51568b4df72a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165514 May 19 12:37:19.276601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1337/2546] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:19.279213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:19.279907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:19.286814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.287387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.288133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.288570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.361374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:19.375213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:19.377043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:19.378195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-543f47e6-c816-44bc-809e-2e0943247108 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.103361 May 19 12:37:19.378777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1210/2547] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:37:19.454183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:19.455168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:19.465719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.466446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.467668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:19.468376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:19.608198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:19.620332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4416e828-33df-4fb3-958b-240cb1e31ab8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172432 May 19 12:37:19.621193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1338/2548] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:20.297308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:20.298379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:20.309332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.310060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.311281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.311976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.396819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:20.397349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:20.404476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.404966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.405720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.406151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.447692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:20.455800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3a9fa48a-e88d-4f70-a5e2-a61dce11794f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165105 May 19 12:37:20.456404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1211/2549] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:20 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:20.467503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:20.476109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:20.477633 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:20.478772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-21f73e2c-b2ac-40c9-8ef5-51857e353839 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.085984 May 19 12:37:20.479321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1339/2550] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:20.643682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:20.644699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:20.655112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.655863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.657426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:20.658225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:20.794304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:20.802872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d49a3e6c-3fb1-43b9-950a-c1ad226cb2b0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165772 May 19 12:37:20.803448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1212/2551] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:21.476831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:21.477884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:21.487707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.488143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.488921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.489341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.500044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:21.500044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:21.504199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.505221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.505461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.505897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.568543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:21.577613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:21.579097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:21.580244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-54541a96-e7a7-4547-b71d-1fa6467fb62d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087693 May 19 12:37:21.580958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1213/2552] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:21.617650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:21.626204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fd2bc59a-dc93-43b1-bd4a-47bbd392ba41 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155785 May 19 12:37:21.626766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1340/2553] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:21.822752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:21.823592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:21.834362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.835104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.836350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:21.837032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:21.994360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:22.007471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-acf362ad-9edd-48f7-b1ff-bc8538839946 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.190831 May 19 12:37:22.008194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1214/2554] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:22.602822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:22.603810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:22.614892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:22.615626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:22.616976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:22.617692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:22.643510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:22.644097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:22.650955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:22.651391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:22.652132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:22.652552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:22.671152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:22.682932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:22.693498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:22.695108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:22.698144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c511d38c-9a16-4bbe-8010-c80eae122b91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100041 May 19 12:37:22.701396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1341/2555] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:22.792673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:22.801064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3538ce7c-6560-4f9e-a070-ea81d55f9ebc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161477 May 19 12:37:22.801608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1215/2556] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:22 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:23.030679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:23.031445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:23.040760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.041283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.042371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.042949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.188874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:23.197032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f96688a4-90d5-45d5-9746-ec6cd4151dbb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172457 May 19 12:37:23.197602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1342/2557] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:23.720080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:23.721266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:23.732495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.733283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.734621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.735369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.819291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:23.819758 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:23.819881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:23.826849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.827287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.828029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:23.828448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:23.828671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:23.830284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:23.831455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-63246fe4-9884-47ef-abae-c4d35eb82eea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.117812 May 19 12:37:23.832006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1216/2558] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:23.960664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:23.968831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b379412-94a9-45b0-8edc-49b64af21bb6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153637 May 19 12:37:23.969401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1343/2559] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:23 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:24.217200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:24.218204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:24.228405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.229157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:24.230439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.231145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:24.377415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:24.390100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d22f1700-6db4-4e61-8e8b-26f7b9f808ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.179102 May 19 12:37:24.390870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1217/2560] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:24.850672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:24.851820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:24.862350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.863100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:24.864351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.865112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:24.933904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:24.942669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:24.944164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:24.945332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2b961858-e81e-43f5-b1a2-32b93a0c13ed tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099504 May 19 12:37:24.946108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1344/2561] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:24.987103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:24.987720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:24.995208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.995654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:24.996455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:24.996927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:25.129016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:25.137135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0c70883a-d756-4f04-87c3-eabe313509ba tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153983 May 19 12:37:25.137708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1218/2562] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:24 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:25.415384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:25.416251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:25.427306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:25.428043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:25.429329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:25.430049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:25.573288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:25.582288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-52f9c5f0-06ea-4eb3-a40c-64c3d47a2dc6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173586 May 19 12:37:25.582903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1345/2563] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:37:25.963713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:25.964318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:25.970996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:25.971443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:25.972200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:25.972633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:26.037527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:26.046163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:26.047917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:26.048899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d15b7c93-ef57-4fe4-9bde-95c67839dc70 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089165 May 19 12:37:26.049475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1219/2564] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:25 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:26.160410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:26.161009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:26.168278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:26.168743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:26.169667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:26.173346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:26.332064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:26.340734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bb56188b-6334-4c3b-a923-cb9d63adee93 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.188417 May 19 12:37:26.341338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1346/2565] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:26 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:26.602704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:26.603695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:26.612672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:26.613137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:26.613879 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:26.614300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:26.752674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:26.761379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c19cc73-15a7-4839-9ae4-a3e801b694e9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165004 May 19 12:37:26.761887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1220/2566] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:26.801508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:37:26.801814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:37:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:37:27.071112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:27.072112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:27.082444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.083167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.084397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.085158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.107064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:27.155712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:27.167107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:27.168803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:27.170262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ebfe5690-c3a1-4d38-88fa-708dad13478d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.105537 May 19 12:37:27.171098 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1347/2567] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:27.361966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:27.362612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:27.370720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.371210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.372457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.372991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.510327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:27.518678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-10c18a78-d57c-4dc3-889f-844f36f2667d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163023 May 19 12:37:27.519236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1221/2568] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:27.782541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:27.783113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:27.790227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.790712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.791665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:27.792139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:27.952330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:27.961409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-defa7120-29b5-4634-8c91-e5151e2419e8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.185439 May 19 12:37:27.961984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1348/2569] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:28.193463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:28.194444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:28.204441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.204981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:28.205721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.206144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:28.277725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:28.286948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:28.288448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:28.289623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-642dc595-9295-45a5-a055-60ed1cdd5c8e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102939 May 19 12:37:28.290181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1222/2570] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:28.537159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:28.537748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:28.544667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.545136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:28.545875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.546296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:28.686204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:28.694526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18b4c985-522a-456b-bed1-f34f9756568b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.161357 May 19 12:37:28.695126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1349/2571] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:28 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:28.979271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:28.979897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:28.986899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.987343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:28.988102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:28.988528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:29.129724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:29.138304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-341b88d3-fdf4-450a-9686-4015c5671eca tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163006 May 19 12:37:29.138888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1223/2572] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:29.308864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:29.309507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:29.316499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:29.316965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:29.317711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:29.318132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:29.382965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:29.392139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:29.393650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:29.394806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-37eebaed-e6b9-436e-980f-e9e07674dc2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090278 May 19 12:37:29.395362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1350/2573] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:29.718921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:29.719823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:29.722768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=80 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:29.731827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:29.732575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:29.733892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:29.734599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:29.875262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:29.883799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cbdd0e99-9809-4348-872c-f4101caedd32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171339 May 19 12:37:29.884370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1224/2574] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:29 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:30.158949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:30.159631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:30.167603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.168118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:30.169026 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.169521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:30.303759 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:30.312163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2b8ffe22-def7-4da6-97ca-17ab44a592c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159364 May 19 12:37:30.312792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1351/2575] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:30.416227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:30.417085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:30.426070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.426513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:30.427261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.427803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:30.501832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:30.510678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:30.512164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:30.513351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-37e1e53c-157a-4122-8e34-fa873598ed78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.103223 May 19 12:37:30.513927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1225/2576] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:30.956045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:30.957124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:30.960025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=78 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:30.968842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.969904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:30.971150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:30.971872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:31.224150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:31.232919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-87fc02e5-e58b-4938-93da-c280b27ef9a1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.283391 May 19 12:37:31.233485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1352/2577] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:30 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:31.332914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:31.333857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:31.343529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:31.344259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:31.345214 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:31.345649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:31.492684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:31.501934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b9920b6d-8574-416c-ae0c-775aa627546c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.175666 May 19 12:37:31.502521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1226/2578] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:31.531463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:31.532127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:31.538298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:31.538762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:31.539461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:31.539881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:31.611093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:31.619995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:31.621573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:31.623028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c0970f9f-ea22-4d25-b032-b8921bad4b78 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095572 May 19 12:37:31.623590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1353/2579] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:32.255050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:32.256046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:32.266556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.267288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.268517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.269271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.415941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:32.426728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-defc164b-5d61-419b-af2d-cad71b30c834 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.178701 May 19 12:37:32.427292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1227/2580] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:32 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:32.519318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:32.519930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:32.526854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.527290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.527994 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.528844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.640418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:32.641045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:32.647493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.647971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.648738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:32.649204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:32.658652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:32.666801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f90ba879-dd30-4a22-919e-91074ef314c9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.151463 May 19 12:37:32.667370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1354/2581] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:32.714553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:32.723222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:32.724692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:32.725845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-97ba2040-4bf3-4b6e-8ed3-10404ed8fbcc tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089440 May 19 12:37:32.726367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1228/2582] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:32 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:33.447885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:33.449017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:33.457333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.457805 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.458546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.458965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.590525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:33.599955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7abdcf26-d40d-4197-acd1-cae65980fe34 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158394 May 19 12:37:33.600519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1355/2583] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:33 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:33.687286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:33.688363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:33.695307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.695747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.696489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.696942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.748268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:33.748899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:33.755262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.755847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.756601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:33.757059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:33.822370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:33.831186 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:33.832689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:33.833823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7e79a390-a561-468a-8f79-34cef1148c31 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089778 May 19 12:37:33.834415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1356/2584] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:33 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:33.838557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:33.846788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e4ba7c75-6944-49bb-9f30-430fbe6a1a8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165952 May 19 12:37:33.847369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1229/2585] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:34.621740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:34.623117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:34.631214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.631656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.632400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.632859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.769624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:34.777909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c2d5ec6-7239-4d88-abd7-088158b0732f tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163518 May 19 12:37:34.778475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1357/2586] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:34 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:34.856111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:34.857167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:34.864867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:34.865066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.865238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.865458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:34.866032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.866468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.871655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.872431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.873210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:34.873630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:34.928787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:34.937686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:34.939263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:34.940492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6e438b8c-a6e2-49d2-a4d8-b68476f48e35 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091059 May 19 12:37:34.941114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1230/2587] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:34 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:35.001836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:35.010067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b68beba3-a4c6-4deb-a77d-17952d5bd456 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.149052 May 19 12:37:35.010637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1358/2588] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:35.798839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:35.799411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:35.806548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:35.807040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:35.807869 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:35.808343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:35.954322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:35.962018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:35.962894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:35.966177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d55ddc32-4806-4bf7-9784-65891de8cc8b tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.173408 May 19 12:37:35.967056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1231/2589] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:35 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:35.971180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:35.971619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:35.972363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:35.972815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:36.027275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:36.027882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:36.034298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:36.034762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:36.035534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:36.035953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:36.040161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:36.041084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:36.050456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:36.051846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:36.053037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d69d9cc-be0d-4b9c-9222-4c16dfd484f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097197 May 19 12:37:36.053605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1359/2590] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:35 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:36.167807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:36.178351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e41692ed-83d6-4576-8105-2b1a3a949020 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154981 May 19 12:37:36.178940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1232/2591] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:37:36.987386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:36.988115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:36.996674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:36.997556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:36.998475 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:36.998999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:37.071278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:37.071855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:37.078606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:37.079140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:37.079887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:37.080307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:37.141011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:37.144304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:37.149088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6b3fdf6e-8d30-433a-97b4-63750f1e6a99 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167568 May 19 12:37:37.149645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1360/2592] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:36 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:37.153385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:37.154858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:37.156241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-521d854e-c949-4f9d-83ce-3fa34f66135a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.088888 May 19 12:37:37.156829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1233/2593] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:37 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:37:37.195566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:37.196171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:37.202249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:37.202689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:37.203429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:37.203943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:37.336001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:37.346982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8e9c0c7a-7fd6-42a7-bddd-87d6b32c8dc5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155570 May 19 12:37:37.347594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1361/2594] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:38.170532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:38.171467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:38.177954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:38.178961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:38.182843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.183623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.184863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.185830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.188847 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.189297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.190046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.190458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.256314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:38.265781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:38.267322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:38.268436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-abd3597f-bdaa-4e0d-88c3-9616363bb652 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097207 May 19 12:37:38.269283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1362/2595] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:38 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:38.322976 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:38.333097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-544d31fd-4c58-4d28-b3ab-b8c70685cd8d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168868 May 19 12:37:38.333703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1234/2596] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:38 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:38.371129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:38.371740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:38.378753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.379189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.379939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:38.380368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:38.511998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:38.520260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4be89b18-23da-42b4-a520-b12811e71b95 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160715 May 19 12:37:38.520862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1363/2597] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:39.292480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:39.293448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:39.302089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.302528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.303274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.303696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.359452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:39.360237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:39.366845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.367263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.368234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.368516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.370751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:39.373331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:39.380466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:39.382017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:39.383174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6d6a8f27-7449-4fc2-a6e2-a2c3b9a73e85 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097183 May 19 12:37:39.383797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1235/2598] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:39 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:39.508676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:39.519574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-964e459b-15ca-470a-a1d3-0fc3c106a14c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164328 May 19 12:37:39.520169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1364/2599] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:39 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:39.537653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:39.538254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:39.544817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.545010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.545797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:39.546265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:39.674278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:39.682368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e898a355-b26a-4a42-a0d8-22b88e8c208a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.148814 May 19 12:37:39.682937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1236/2600] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:40.405695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:40.406634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:40.414003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.414447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.415182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.415632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.537024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:40.537613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:40.543415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:40.544221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.544678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.545429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.545849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.551962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:40.553497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:40.554640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fe7ae5a3-08f8-4709-8a72-87f2318cf8de tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.155562 May 19 12:37:40.555202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1365/2601] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:40 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:40.676203 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:40.684281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cd6b6da0-ee9c-4275-b412-bf8319eb2c0e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.151564 May 19 12:37:40.684871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1237/2602] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:40 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:40.826697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:40.827551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:40.834065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.834511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.835255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:40.835681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:40.971654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:40.980015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4a04342d-f3cd-4977-893e-fff280f9c0bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.284286 May 19 12:37:40.980673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1366/2603] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:40 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:41.573374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:41.574020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:41.581149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:41.581591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:41.582336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:41.582775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:41.701001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:41.701607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:41.708610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:41.709897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:41.710688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:41.711152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:41.711698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:41.720445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:41.721983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:41.723130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-25ec8fb2-182a-46c4-b0a8-ad4000554805 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.153738 May 19 12:37:41.723676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1238/2604] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:41 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:41.843586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:41.852516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cb2ef5ae-97a7-4620-8942-3274641f54c0 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155354 May 19 12:37:41.853119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1367/2605] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:41 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:42.000777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:42.001494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:42.008425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.008967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:42.009791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.010257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:42.141510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:42.153710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f00e2fdf-38a1-4fdb-a260-f592e3014db6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159458 May 19 12:37:42.154455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1239/2606] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:42.745116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:42.746133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:42.755438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.755858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:42.756618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.757079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:42.823661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:42.832488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:42.834025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:42.835171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-516e8d12-8aa9-4660-bb50-9dbe8d1a0fc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096502 May 19 12:37:42.835733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1368/2607] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:42 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:42.869695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:42.870316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:42.877599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.878040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:42.878781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:42.879198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:43.033793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:43.042385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bd163d68-5be1-4c5d-b98e-7329f3337f46 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.176536 May 19 12:37:43.042972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1240/2608] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:42 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:43.174494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:43.175010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:43.181493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:43.181936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:43.182688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:43.183110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:43.317794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:43.325800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c8b7cea2-391b-43e0-a1ea-1b915998051f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.156903 May 19 12:37:43.326394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1369/2609] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:43.857558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:43.858548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:43.865834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:43.866312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:43.867092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:43.867521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:43.929425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:43.938339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:43.940219 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:43.941380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bc83e93a-aa42-4930-a4d1-e813c92e306d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090445 May 19 12:37:43.941947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1241/2610] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:43 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:44.064939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:44.065527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:44.072632 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.073105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:44.073852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.074276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:44.203424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:44.215549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-16e6ceec-e3ad-4e96-9541-04c6764d66dc tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.155957 May 19 12:37:44.216303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1370/2611] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:44 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:44.345640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:44.346269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:44.353143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.353605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:44.354383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.354836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:44.497995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:44.508243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-34507abe-6525-4ce5-a626-a3b47304547c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167534 May 19 12:37:44.508243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1242/2612] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:44.963192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:44.964195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:44.972768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.973214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:44.973963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:44.974394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:45.039673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:45.048895 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:45.050370 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:45.051515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b4c56b53-b8af-4269-9ce9-5418dc06b705 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094931 May 19 12:37:45.052081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1371/2613] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:44 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:45.237294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:45.237850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:45.244872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:45.245327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:45.246146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:45.246555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:45.382143 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:45.390158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-035a749b-dfcc-405c-b5a3-7aab066a7b9c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158667 May 19 12:37:45.390727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1243/2614] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:45 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:45.525214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:45.525821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:45.532515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:45.533002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:45.533749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:45.534191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:45.677496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:45.685527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ae785187-002d-4b3e-a945-fd33782992c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164259 May 19 12:37:45.686116 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1372/2615] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:46.073935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:46.074931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:46.082997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.083442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.084238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.084675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.154483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:46.166575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:46.168060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:46.169455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-00e5398f-5ba2-4023-8674-4cc041f485c6 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101841 May 19 12:37:46.170310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1244/2616] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:46 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:46.410446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:46.411104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:46.418182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.418649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.419466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.419929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.560960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:46.569780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5bf1291-353a-4f41-ab68-2f61b9d18d01 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164855 May 19 12:37:46.570626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1373/2617] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:46 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:46.702242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:46.702826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:46.709585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.710029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.710788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:46.711216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:46.846014 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:46.858207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dc369f6f-98f6-4691-92b4-90535aa7228a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159530 May 19 12:37:46.858948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1245/2618] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:47.193215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:47.193799 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:47.199917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.200367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:47.201117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.201541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:47.267631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:47.281149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:47.283027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:47.284173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-430d83a9-3b37-4d11-a615-cf7ae52931f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095019 May 19 12:37:47.284749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1374/2619] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:47 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:47.591327 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:47.592307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:47.602593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.603313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:47.604689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.605387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:47.743821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:47.752503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-67feecd4-06cd-4158-ac10-8447a5530de9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167666 May 19 12:37:47.753119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1246/2620] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:47 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:47.877537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:47.878137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:47.885010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.885449 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:47.886196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:47.886617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:48.025292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:48.033493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35426acd-cde6-49d6-a72e-f4e9acb7a819 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161416 May 19 12:37:48.034057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1375/2621] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:48.306528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:48.307338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:48.319163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:48.319905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:48.321416 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:48.322118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:48.386620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:48.395397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:48.396911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:48.398044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6672b02a-9c7c-4fe2-8dcf-02a7997d006e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.098376 May 19 12:37:48.398595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1247/2622] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:48 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:48.774173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:48.775160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:48.784277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:48.784748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:48.785487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:48.785903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:48.916722 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:48.926165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dd034b53-0483-49e0-85e4-084260ee1330 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.158621 May 19 12:37:48.926710 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1376/2623] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:48 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:49.055024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:49.056117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:49.066340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.067069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:49.068343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.069099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:49.206738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:49.210144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:49.215401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9675521b-3f10-4292-b25f-482b7b9b7187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167065 May 19 12:37:49.216077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1248/2624] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:49.424467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:49.425549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:49.435699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.436436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:49.437721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.438464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:49.503132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:49.515152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:49.517621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:49.519387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b52e3def-ffcd-46f2-90be-b1cc3f737416 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.101280 May 19 12:37:49.520260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1377/2625] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:49 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:49.944484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:49.945175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:49.952680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.953191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:49.954102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:49.954533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:50.091929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:50.100829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2f6ff3ff-d5bd-466a-a22d-bf9de0f50d38 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160926 May 19 12:37:50.101370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1249/2626] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:49 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:50.240797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:50.241611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:50.252208 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:50.252672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:50.253434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:50.253890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:50.393711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:50.402355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6ddde2c3-3c01-497b-95cc-f374c517d1d6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168075 May 19 12:37:50.402905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1378/2627] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:50.545052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:50.545897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:50.553350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:50.553794 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:50.554532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:50.554953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:50.618682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:50.631974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:50.634315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:50.635915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cdbc88db-9468-4457-8f9f-c261e3ffcd1b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096860 May 19 12:37:50.636745 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1250/2628] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:50 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:51.122510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:51.123438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:51.130869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.131301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.132038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.132459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.265308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:51.273380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0969e4dd-0692-4aa5-9fbd-e17a6ecdcb08 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157788 May 19 12:37:51.273947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1379/2629] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:51 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:51.423419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:51.423979 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:51.430942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.431388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.432141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.432569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.575101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:51.587785 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f67e4123-c9b2-480f-8bdd-47a59a1c1146 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170355 May 19 12:37:51.588605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1251/2630] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:51.653753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:51.654408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:51.661021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.661495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.662251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:51.662680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:51.728834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:51.733581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:51.743010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:51.745410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:51.747552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-40c67996-47f6-4ace-b5cc-d4eab1924de7 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097205 May 19 12:37:51.748395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1380/2631] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:51 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:52.290093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:52.290698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:52.297206 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.297687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.298427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.298852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.439773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:52.448420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bdd449ce-ec26-40c6-87d1-f828cf1b0add tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.162311 May 19 12:37:52.449036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1252/2632] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:52 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:52.608506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:52.609171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:52.620616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.620616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.621364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.622373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.759270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:52.766932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:52.767496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:52.767674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-22c90964-f1e6-4529-81ab-b29b891c7a61 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165244 May 19 12:37:52.768128 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1381/2633] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:52.773811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.774257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.775008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:52.775436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:52.844113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:52.852665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:52.854151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:52.855293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b43d9990-b2f8-4c48-9dc5-7143e790a0f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092457 May 19 12:37:52.855938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1253/2634] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:52 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:53.469528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:53.471330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:53.481310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.481988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.483211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.483894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.621500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:53.629990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f2f9d8b-83fe-4759-8647-f4528d943c62 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166680 May 19 12:37:53.630552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1382/2635] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:53 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:53.785034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:53.785620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:53.792995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.793463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.794252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.794677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.874124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:53.882720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:53.890591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.891040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.891808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:53.892243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:53.934463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:53.945089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b8f1d9a3-1d8a-4aba-9a0c-1b90c806efde tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164191 May 19 12:37:53.945698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1254/2636] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:53.958503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:53.967083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:53.968578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:53.969830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cde930a8-940c-4458-ac77-14e2d2e74812 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100328 May 19 12:37:53.970395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1383/2637] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:53 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:54.654866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:54.655914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:54.662919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.663371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:54.664104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.664547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:54.800407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:54.809006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1f744fe5-9866-49d9-a254-9824d5d70a9a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160770 May 19 12:37:54.809581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1255/2638] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:54 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:54.966138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:54.967138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:54.978350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.979139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:54.980366 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.981177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:54.989121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:54.989723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:54.996268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.996706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:54.997789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:54.998224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:55.063796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:55.073997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:55.076056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:55.077806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fe4bc438-cd40-49a2-be47-252a3af5c483 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092873 May 19 12:37:55.078654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1256/2639] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:54 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:55.119756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:55.131973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d78d22ee-42fe-4618-bab9-4bd9d9cd64b3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172067 May 19 12:37:55.132788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1384/2640] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:55.830912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:55.831607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:55.840348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:55.840947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:55.841884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:55.842414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:55.985748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:55.994259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-433844ac-1d8f-4d8f-a9bf-76844b24fa22 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169786 May 19 12:37:55.994841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1257/2641] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:55 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:56.101575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:56.102222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:56.112883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:56.113487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:56.114620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:56.115108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:56.152101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:56.152734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:56.159299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:56.159750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:56.160546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:56.161011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:56.180503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:56.189061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:56.190665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:56.191796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd37810a-5863-421e-bc25-66b2f747c031 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.096028 May 19 12:37:56.192355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1385/2642] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:56 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:56.291111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:56.299766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bf3cc549-bd39-461c-9932-b89833ab3c9e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.151496 May 19 12:37:56.300348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1258/2643] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:57.016085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:57.017124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:57.027505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.028239 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.029522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.030225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.171604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:57.179997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b35e607-e534-4ff7-b0dd-cbbcd449cf2a tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.170295 May 19 12:37:57.181077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1386/2644] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:57 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:57.210121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:57.210727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:57.217489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.217924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.218683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.219109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.289130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:57.297765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:57.299254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:57.301902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b6970ac6-1357-4e0d-879c-86011014db29 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095778 May 19 12:37:57.303625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1259/2645] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:57 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:57.319337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:57.320179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:57.329204 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.329806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.330941 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:57.331648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:57.463769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:57.475233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-64847118-6c74-44a9-b25c-5c230d4ef564 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161307 May 19 12:37:57.475846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1387/2646] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:58.201195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:58.202165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:58.210455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.211053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.211646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.212072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.322210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:58.322800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:58.329638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.330094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.330854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.331283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.356353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:58.364743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-88132257-c9d7-4f9c-aba0-00898ce01893 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169835 May 19 12:37:58.365324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1260/2647] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:58 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:58.406851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:58.416313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:58.417839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:58.418984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1dbe557b-4b82-4bfb-911f-fbc026e0cc54 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.100826 May 19 12:37:58.419541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1388/2648] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:58 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:58.498420 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:58.499062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:58.505685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.506119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.506864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:58.507289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:58.640872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:58.650612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-21de562a-b841-4a5b-9cb7-e423aaf26529 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.156538 May 19 12:37:58.651177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1261/2649] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:59.386382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:59.387510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:59.397650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.398096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.398855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.399281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.437606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:59.438350 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:59.446247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.446691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.447636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.448054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.514533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:59.522961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:59.524448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:59.525630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a7b15f77-faaa-44e0-9bdc-70f4bb50ca92 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092131 May 19 12:37:59.526189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1262/2650] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:37:59 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:37:59.534858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:59.543018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa7bf44c-63b7-499e-acad-783d4299f60c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.163135 May 19 12:37:59.543578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1389/2651] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:59 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:37:59.671804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:37:59.672704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:37:59.680563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.681046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.681793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:37:59.682222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:37:59.812576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:37:59.820933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a8b9dc4-273c-42dc-afab-f3dad9f9bd26 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155536 May 19 12:37:59.821492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1263/2652] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:37:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:00.543567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:00.544151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:00.551229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.551699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.552455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.552935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.559720 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:00.560298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:00.562845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=80 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:00.567960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.568401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.569183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.569609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.617689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:00.626875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:00.628357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:00.629586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e51782df-66a3-4bf0-ba4c-2749813753ee tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.090006 May 19 12:38:00.630153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1390/2653] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:00 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:00.701719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:00.709796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bb43f1f4-94d2-476b-bdae-ad9cec10b8ac tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.154428 May 19 12:38:00.710400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1264/2654] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:00 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:00.841887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:00.842883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:00.853272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.854008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.855245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:00.856008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:00.999249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:01.011547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e078bbd-072d-46e3-af06-d67ac2b655b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.175856 May 19 12:38:01.012358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1391/2655] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:01.650059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:01.650737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:01.658953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:01.659522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:01.660428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:01.661033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:01.682099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:01.727357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:01.728022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:01.730692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=80 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:01.733584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:01.738626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:01.739302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:01.740255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:01.740686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:01.744321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:01.745893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:01.747392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6551b75e-0383-4337-90fe-62b8a21d10cb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.102863 May 19 12:38:01.747946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1265/2656] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:01 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:01.886336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:01.894688 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8c3c5204-a30d-4d3f-af5c-d0092f89c6b4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.171225 May 19 12:38:01.895250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1392/2657] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:01 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:02.031986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:02.032595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:02.038817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.039258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:02.039995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.040418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:02.189931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:02.198554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a14a9c2b-43e0-4142-9959-3d30f4253fb5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170468 May 19 12:38:02.199122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1266/2658] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:02.769775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:02.770733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:02.781562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.782283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:02.783494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.784196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:02.847621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:02.856280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:02.857786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:02.858912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5cd9c779-7a5b-4ec8-8152-56496c09927f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095863 May 19 12:38:02.859476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1393/2659] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:02 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:02.912348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:02.912953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:02.920108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.920541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:02.921286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:02.921705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:03.049645 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:03.061791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5c86120a-0c1c-458c-b3c8-c712ff6c6030 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153110 May 19 12:38:03.062686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1267/2660] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:02 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:03.219332 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:03.220326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:03.230885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:03.231662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:03.232960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:03.233665 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:03.374062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:03.382418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-57177730-2ea5-4b14-b113-63ab4778a8da tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169482 May 19 12:38:03.382990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1394/2661] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:03.885087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:03.888826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:03.900379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:03.900379 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:03.901611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:03.902564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:03.975537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:03.984561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:03.986173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:03.987321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8b3edffb-fa29-47f8-bbbf-2681d6e28f8b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.113234 May 19 12:38:03.987875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1268/2662] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:03 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:04.083642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:04.084259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:04.091463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:04.091913 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:04.092676 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:04.093111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:04.269271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:04.277830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b95754bf-b9dd-4e0f-94c4-180052dc959e tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.200898 May 19 12:38:04.278395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1395/2663] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:04 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:04.403587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:04.404515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:04.413518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:04.413966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:04.414725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:04.415150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:04.559993 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:04.569830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae472a23-1795-407d-9287-3c5f83371854 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172322 May 19 12:38:04.570982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1269/2664] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:05.010288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:05.011318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:05.021709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.022434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.023806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.024514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.092514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:05.101043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:05.102515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:05.103649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e53c32de-72ab-4acd-bef7-13f1df119f37 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099948 May 19 12:38:05.104203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1396/2665] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:05 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:05.301983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:05.302789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:05.313854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.314588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.315825 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.316534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.452358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:05.460956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bc1a6cd3-7ad7-4216-9278-9ab3d4ba6d66 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165499 May 19 12:38:05.461537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1270/2666] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:05 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:05.598348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:05.599298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:05.609018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.609707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.610858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:05.611514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:05.631422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:05.747434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:05.755945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8bdd830f-6937-4deb-becc-0c73a88fabfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163598 May 19 12:38:05.756482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1397/2667] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:06.124980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:06.125807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:06.134851 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.135473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.136549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.137118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.211641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:06.223918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:06.225980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:06.227616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ba9a5f07-da67-43a5-b42b-3f150eff2ed2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.109234 May 19 12:38:06.228363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1271/2668] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:06 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:06.482640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:06.483467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:06.494570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.496066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.498004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.498729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.645649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:06.653776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7fc18b1f-4ce6-4be5-9000-f6bd9277b005 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.177813 May 19 12:38:06.654316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1398/2669] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:06 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:06.775567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:06.776281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:06.785321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.785908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.786839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:06.787261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:06.922772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:06.930915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9814dd5e-df44-4f69-9ba6-58c7a94f377d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160598 May 19 12:38:06.931455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1272/2670] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:07.250919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:07.251514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:07.258069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.258512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:07.259253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.259690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:07.320742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:07.329741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:07.331231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:07.332375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92dfa487-d106-4995-a527-5192426bc9b4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.085800 May 19 12:38:07.332962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1399/2671] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:07 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:07.677390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:07.677994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:07.684580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.685057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:07.685805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.686230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:07.815062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:07.823377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab25389b-ac96-4807-ac6f-f58825c9d94d tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.152445 May 19 12:38:07.823949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1273/2672] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:07 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:07.951432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:07.952375 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:07.962647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.963373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:07.964594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:07.965352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:08.101150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:08.109240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ffb89b6-6b8b-4f6a-9285-f13195812877 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164414 May 19 12:38:08.109781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1400/2673] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:08.350490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:08.351197 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:08.358727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:08.359171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:08.359923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:08.360380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:08.434278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:08.442994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:08.444473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:08.445635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8c1b127d-4250-4766-a6f2-c1eeadcb8428 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099495 May 19 12:38:08.446194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1274/2674] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:08 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:08.844860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:08.845851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:08.853910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:08.854352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:08.855108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:08.855506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:08.995703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:09.004425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ba0e7f1-ef2b-4d59-8bd1-30901ad1e508 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166142 May 19 12:38:09.005148 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1401/2675] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:08 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:09.130067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:09.130922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:09.142059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:09.142803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:09.144043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:09.144796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:09.287183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:09.295559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fd69b368-3914-449f-ad36-fbcb22d13e4a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172017 May 19 12:38:09.296133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1275/2676] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:09.467376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:09.468241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:09.476873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:09.477348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:09.478092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:09.478515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:09.545120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:09.553703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:09.555181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:09.556311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-19e3a74f-c5f7-4080-b2b3-1696f4e6b2a5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095731 May 19 12:38:09.556924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1402/2677] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:09 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:10.022823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:10.023569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:10.030947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.031464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.032328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.032886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.167247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:10.175479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0411782f-34b7-433d-8e50-c27d52cb6f32 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.157257 May 19 12:38:10.176097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1276/2678] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:10 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:10.315288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:10.316144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:10.323554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.324008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.324793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.325203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.462561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:10.470707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b8630f46-0ba8-42a2-b8c8-32b601f41b40 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161065 May 19 12:38:10.471277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1403/2679] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:10.578817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:10.579696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:10.587495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.587929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.588668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:10.589117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:10.664070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:10.677471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:10.678965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:10.680084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-46fb2428-a907-401e-959e-29b6c0ed954a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.107975 May 19 12:38:10.680695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1277/2680] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:10 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:11.196284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:11.197122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:11.205288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.205876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.206865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.207432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.343494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:11.355586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7b11cee3-7a7d-447c-9cd6-078536b9cc70 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.165393 May 19 12:38:11.356376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1404/2681] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:11 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:11.491534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:11.492402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:11.502567 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.503166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.504181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.504797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.647718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:11.655841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4daa53aa-bc27-4c41-beae-28f9dac2b629 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170367 May 19 12:38:11.656413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1278/2682] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:11.698030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:11.698624 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:11.705298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.705733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.706480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:11.706857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:11.771892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:11.780381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:11.781888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:11.783015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f56dbcf1-676a-4a90-a9fd-c6edde2531fe tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.088908 May 19 12:38:11.783575 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1405/2683] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:11 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:12.379317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:12.380297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:12.389788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.390513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.391710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.392365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.537183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:12.545523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b932f540-b1fd-4f15-bd3b-4d2cafd48ab3 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.172584 May 19 12:38:12.546095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1279/2684] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:12 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:12.677040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:12.678028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:12.689374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.690114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.691379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.692097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.801465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:12.801986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:12.808386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.808878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.809653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:12.810077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:12.831439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:12.839642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d9f913a7-49ce-486c-89c7-9db74904878e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168962 May 19 12:38:12.840192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1406/2685] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:12.873907 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:12.882528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:12.883995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:12.885160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3e54ed1c-96cd-47f3-b6c0-25d484275bc4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.087813 May 19 12:38:12.885723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1280/2686] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:12 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:13.573391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:13.574316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:13.584495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.585154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.585906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.586341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.718417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:13.727092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-77be2fc8-4f74-41ad-bbe8-68011d8e13b6 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160378 May 19 12:38:13.727658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1407/2687] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:13 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:13.860931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:13.861816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:13.870465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.870914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.872241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.872736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.904852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:13.906392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:13.913039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.913729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.914516 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:13.914936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:13.981020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:13.989660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:13.991133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:13.992292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f27f4f28-f3f7-419f-a376-ce5e7081c482 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092545 May 19 12:38:13.992861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1408/2688] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:13 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:14.007633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:14.015930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f15530a6-689b-44f6-935b-98050d09c272 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.161718 May 19 12:38:14.016497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1281/2689] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:14.748727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:14.749744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:14.759277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:14.759743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:14.760479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:14.760934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:14.901033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:14.909642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ea6aa411-0996-489b-820d-b92735ddecf1 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167901 May 19 12:38:14.910214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1409/2690] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:14 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:15.014109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:15.015110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:15.024877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.025320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:15.026066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.026495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:15.030498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:15.032139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:15.032776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:15.039178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.039638 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:15.040376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.040842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:15.088404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:15.096973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:15.098508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:15.099543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-041bcff6-d05c-4ad8-8f13-119917d6ccf9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.092031 May 19 12:38:15.100122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1282/2691] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:15 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:15.170876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:15.179333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4dea15b0-91cf-48e1-b7db-c50e902b49f6 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.151144 May 19 12:38:15.179904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1410/2692] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:15.930896 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:15.931598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:15.939257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.939778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:15.940688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:15.941191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:16.094538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:16.106654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c65ca7a-0f38-44d4-ac86-50b212f72d76 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.181342 May 19 12:38:16.107352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1283/2693] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:15 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:16.116894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:16.117487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:16.124742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:16.125183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:16.125930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:16.126348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:16.196178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:16.197266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:16.197928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:16.204086 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:16.204526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:16.205196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:16.205317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:16.205714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:16.206681 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:16.207813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1a659397-4372-4284-90e5-ec22aa631d77 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094919 May 19 12:38:16.208407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1411/2694] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:16 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:16.347240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:16.355413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-997285b2-a8e5-4d14-878d-5786d6978755 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162774 May 19 12:38:16.356044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1284/2695] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:17.128996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:17.129924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:17.137822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.138263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.139010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.139435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.226227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:17.226833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:17.233984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.234436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.235182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.235609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.273304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:17.281506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-56126d7a-f8f3-4310-92fc-facee1f7e8bd tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.159138 May 19 12:38:17.285924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1412/2696] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:17 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:17.297229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:17.305752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:17.307225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:17.308382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c85c902b-349b-4d03-bf6b-2f2efbd3a701 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086299 May 19 12:38:17.308959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1285/2697] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:17 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:17.377087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:17.378087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:17.388541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.389237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.390478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:17.391203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:17.528674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:17.537042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-be92c7a1-6957-4d2a-a2a2-1d7fb061fccb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166983 May 19 12:38:17.537617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1413/2698] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:18.302822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:18.303721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:18.315166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.315767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.317917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.318470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.329591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:18.330222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:18.337351 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.337765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.338518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.338939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.343071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:18.405326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:18.414157 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:18.415651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:18.416824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18ddd2df-4375-4968-86bb-2bf5ff90089c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091090 May 19 12:38:18.417384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1414/2699] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:18 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:18.452757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:18.461000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c569d2d-db4a-4220-acef-ff83ed0866bb tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164648 May 19 12:38:18.461601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1286/2700] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:18 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:18.558763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:18.559752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:18.569696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.570133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.570869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:18.571290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:18.711719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:18.720138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b1acf175-81de-4cb6-9f12-c3e4a66c5d10 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168365 May 19 12:38:18.720704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1415/2701] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:19.439374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:19.440306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:19.448736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.449172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.449912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.450330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.477888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:19.478481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:19.485507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.485954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.486689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.487112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.512980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:19.521578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:19.523103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:19.524237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a459bcd0-24c4-4da7-9a09-1743407ec9ea tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.091447 May 19 12:38:19.524821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1287/2702] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:19 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:19.617200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:19.625677 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f54e003f-0400-45b5-9118-fb6321b22cb9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.151728 May 19 12:38:19.626248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1416/2703] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:19 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:19.740556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:19.741231 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:19.748333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.748820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.749589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:19.750024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:19.893969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:19.903180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-68d72849-43ff-4a57-a421-ab4a1a855ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168524 May 19 12:38:19.903718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1288/2704] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:20.542792 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:20.543393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:20.550522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.550958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:20.551705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.552132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:20.619985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:20.632704 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:20.634956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:20.636802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cd361d80-072c-439e-8f9c-1590a9a479f4 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.097730 May 19 12:38:20.637515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1417/2705] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:20 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:20.643282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:20.643969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:20.650978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.651423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:20.652160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.652585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:20.788258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:20.796425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e7c85e00-9dfc-4c2c-bb4f-5c3b012d6aa4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.156958 May 19 12:38:20.797033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1289/2706] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:20 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:20.924278 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:20.925543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:20.936369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.937266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:20.938517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:20.939235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:21.077929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:21.086330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0a47267-4451-43bf-aa73-e7f4df3d282a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168602 May 19 12:38:21.086908 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1418/2707] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:21.659085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:21.660065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:21.667889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:21.668278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:21.669099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:21.669520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:21.740378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:21.749649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:21.751170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:21.752343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01ca4590-1951-4b55-8e0a-acddd71ca642 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099933 May 19 12:38:21.752942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1290/2708] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:21 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:21.817837 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:21.818842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:21.829888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:21.830626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:21.831882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:21.832912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:21.967985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:21.980482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ad6014af-21fd-4897-ae97-c33b73f172fa tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.168964 May 19 12:38:21.981367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1419/2709] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:21 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:22.108061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:22.108977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:22.118102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:22.118538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:22.119283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:22.119706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:22.252663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:22.261456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bbd69235-0ef1-4442-b87c-8a23f59988b2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159899 May 19 12:38:22.262072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1291/2710] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:22.771678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:22.772220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:22.778725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:22.779162 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:22.779899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:22.780322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:22.851926 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:22.860751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:22.862194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:22.863344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-71250bee-17de-487a-a662-20c31327aa63 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095737 May 19 12:38:22.863903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1420/2711] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:22 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:23.004472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:23.005508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:23.012895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.013325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.014060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.014495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.149821 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:23.157977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c396a5ba-e05e-4072-bf38-83be63874ce4 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.160042 May 19 12:38:23.158580 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1292/2712] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:22 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:23.278810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:23.279435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:23.285961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.286401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.287135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.287560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.416007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:23.424226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cf9007f9-344d-4064-a87c-8e9d410e1cd9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.149348 May 19 12:38:23.424827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1421/2713] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:23.884587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:23.885255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:23.892087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.892571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.893388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:23.893933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:23.957679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:23.970572 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:23.972780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:23.974419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-68bd0418-170f-4a05-92e2-899f1b737557 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.094569 May 19 12:38:23.975241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1293/2714] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:23 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:24.179413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:24.180079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:24.188693 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:24.189275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:24.190194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:24.190717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:24.330564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:24.338939 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b5da4ffd-244f-4286-9310-2f2cb41d4c2c tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.166073 May 19 12:38:24.339507 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1422/2715] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:24 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:24.441657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:24.442201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:24.449158 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:24.449611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:24.450348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:24.450775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:24.585194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:24.597618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aebd25f8-1306-4f76-8d05-c3270e14e9f0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159640 May 19 12:38:24.598455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1294/2716] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:25.000478 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:25.001524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:25.011809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.012585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.014171 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.014901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.081265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:25.089950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:25.091562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:25.092694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bccb67c7-45c0-45c2-a89b-b67705cecfe5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099406 May 19 12:38:25.093261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1423/2717] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:24 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:25.362115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:25.362899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:25.372536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.373170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.374292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.374856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.512744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:25.520793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4e0d604e-7122-41c4-89a5-f078e54f9cf9 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.164740 May 19 12:38:25.521352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1295/2718] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:25 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:25.616846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:25.617538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:25.627140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.627742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.628797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:25.629386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:25.777824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:25.786010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cc237908-640c-4454-a7b6-acf2d348f5d2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.174644 May 19 12:38:25.786576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1424/2719] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:26.114908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:26.115927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:26.126752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.127439 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.128754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.129460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.192165 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:26.200928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:26.202403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:26.203541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-218bf614-5db3-4244-8532-5f8c357ddd91 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095215 May 19 12:38:26.204114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1296/2720] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:26 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:26.538002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:26.538594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:26.545119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.545558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.546294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.546714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.678915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:26.687123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c72c102a-c851-4b95-a35c-2f5144e47ae7 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.153139 May 19 12:38:26.687714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1425/2721] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:26 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:26.803337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:26.803927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:26.810931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.811371 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.812112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:26.812532 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:26.954541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:26.967827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b846a1b-792e-4a89-9b75-7487430f5843 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168189 May 19 12:38:26.968671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1297/2722] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:27.226372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:27.226975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:27.233259 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.233705 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:27.234451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.234875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:27.299972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:27.308917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:27.310395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:27.311530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c8a6023-ec4c-48ed-ae12-ed5a268372d0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.089273 May 19 12:38:27.312092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1426/2723] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:27 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:27.708455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:27.709499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:27.719767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.720494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:27.721772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.722482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:27.862677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:27.866147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:27.871999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e4cb07ad-c77b-47a2-a141-8e379ae458a2 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.169681 May 19 12:38:27.872581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1298/2724] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:27 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:27.988156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:27.988888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:27.996854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.997315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:27.998054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:27.998479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:28.138630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:28.147310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2b77d3fa-3c61-44a7-86fb-0897671e7551 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165032 May 19 12:38:28.147881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1427/2725] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:28.334950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:28.335823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:28.345950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:28.346684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:28.347893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:28.348608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:28.426194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:28.441099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:28.443584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:28.444969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b558dbc5-7522-49a0-8a96-e796ba95f5f3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.117301 May 19 12:38:28.445511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1299/2726] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:28 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:28.893706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:28.894717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:28.902686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:28.903335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:28.904934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:28.905357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:29.046574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:29.054931 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0bff69dd-5cdb-4bbf-8c4b-ea10eecc0400 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.167771 May 19 12:38:29.055605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1428/2727] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:28 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:29.168364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:29.169239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:29.180384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:29.181166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:29.182453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:29.183148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:29.462755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:29.463224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:29.469758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:29.470200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:29.470953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:29.471377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:29.539053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:29.542357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:29.547522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c1c7a77e-6a3e-4a97-94af-991ed0f42641 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.385742 May 19 12:38:29.548193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1300/2728] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:29.551152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:29.552684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:29.553875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e0e5cf94-ed9a-400e-8e9d-cdb88e496c3e tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.095405 May 19 12:38:29.554424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1429/2729] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:30.075891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:30.076924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:30.086154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.086614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.087337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.087779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.233972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:30.242633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1f05a512-7ec9-41a0-a873-bff9372946ca tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.173094 May 19 12:38:30.243140 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1301/2730] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:30 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:30.565266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:30.565824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:30.571474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:30.572074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:30.572974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.573417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.573961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=76 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:30.574140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.574588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.579684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.580123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.580893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:30.581313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:30.642518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:30.651051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:30.652539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:30.653729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e7848f18-bcfb-497f-ad0d-f69fd285237f tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.086231 May 19 12:38:30.654483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1302/2731] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:30.709067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:30.714027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:30.720904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7fa4c8b2-7d47-45f2-9891-edf94eaafc72 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159211 May 19 12:38:30.721668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1430/2732] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:31.265127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:31.266055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] [instance: e2382eba-ee5f-4026-8253-b5835cfb84c4] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:31.277421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.278170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.279415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.280135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.430092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd059ba56941148f3bb6080ba4f537f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92c49142ea324049ba24b55ff84e5a3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:31.438533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b95fc87d-143f-438e-8d51-86735fd96b17 tempest-VolumesSnapshotTestJSON-1547752884 tempest-VolumesSnapshotTestJSON-1547752884-project] 10.222.0.35 "GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4" status: 200 len: 1560 microversion: 2.1 time: 0.179883 May 19 12:38:31.439110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1303/2733] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:31 2022] GET /compute/v2.1/servers/e2382eba-ee5f-4026-8253-b5835cfb84c4 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:31.674641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:31.675503 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:31.684140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.684585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.685360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.685787 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.739713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:31.740313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:31.746842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.747267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.748070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:31.748493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:31.756824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:31.765820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:31.767299 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:31.768487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-72264b5d-93d7-42e1-b116-c6e6cfbf7174 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1445 microversion: 2.20 time: 0.099215 May 19 12:38:31.769090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1431/2734] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:31.788831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8077e8fb-7acb-48b4-b306-5d5b78410e6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '.version_select at 0x7fe20c0649d0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:31.793951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR oslo.messaging._drivers.impl_rabbit [None req-8077e8fb-7acb-48b4-b306-5d5b78410e6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [194144ce-f3a0-4eb1-a085-68077b670c55] AMQP server on 10.222.0.35:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 19 12:38:31.877363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:31.885630 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-73d01220-0835-4a3f-977d-4d8fa9d1dfd4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.150043 May 19 12:38:31.886181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1304/2735] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:32.015144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:32.015619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:32.025548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.026146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:32.027120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.027554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:32.055252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:38:32.136088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-36b8e978-1727-4b9c-ae6a-0c92dfd12dc8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 204 len: 0 microversion: 2.1 time: 0.125722 May 19 12:38:32.136921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1305/2736] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:38:32 2022] DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 19 12:38:32.146259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:32.146956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:32.153299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.153741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:32.154495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.154919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:32.297127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:32.305514 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bd38e5f2-6d91-4c83-8757-f1b52e013265 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1718 microversion: 2.1 time: 0.163429 May 19 12:38:32.306097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1306/2737] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1718 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:32.803343 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [None req-8077e8fb-7acb-48b4-b306-5d5b78410e6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [194144ce-f3a0-4eb1-a085-68077b670c55] Reconnected to AMQP server on 10.222.0.35:5672 via [amqp] client with port 48254. May 19 12:38:32.813519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=80 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:32.830723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8077e8fb-7acb-48b4-b306-5d5b78410e6b tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2125010663" status: 204 len: 0 microversion: 2.20 time: 1.047771 May 19 12:38:32.831312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1432/2738] 10.222.0.35 () {60 vars in 1261 bytes} [Thu May 19 12:38:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2125010663 => generated 0 bytes in 1049 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 19 12:38:32.902117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:32.902761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:32.909240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.909789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:32.910548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:32.910958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:33.086868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:33.094897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-21c95f03-5d97-4e96-a109-8eef588011ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.196666 May 19 12:38:33.095647 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1307/2739] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:33.322728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:33.323334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:33.330205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:33.330650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:33.331426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:33.331850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:33.634790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:33.643120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-58a5458c-4bed-4560-81bf-6e80dbd4edfa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1718 microversion: 2.1 time: 0.324362 May 19 12:38:33.643691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1433/2740] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1718 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:34.072970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-609d2326-3eb4-4b3b-9a67-129e65df2ec3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'create', calling method: .version_select at 0x7fe1fff51940>, body: {"keypair": {"name": "tempest-keypair-1748312208", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEKamNNCr6FsRCxYkoU+/65j7XMHx+sIhPWLbw/KSSqlDbt1Wn095hRDq5DYgxTMy10Im3v0iEhB1jrBCA+uouL7zu/lfXuxHDqfSYJt2ueIPuK4Y9rrYJlNKcroaxY9wg=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:34.089124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-609d2326-3eb4-4b3b-9a67-129e65df2ec3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:38:34.092990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-609d2326-3eb4-4b3b-9a67-129e65df2ec3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for user b4113cb440764819996cfffec45e0b76 and project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:38:34.111977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:34.113399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:34.121150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:34.121906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:34.122979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:34.123109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-609d2326-3eb4-4b3b-9a67-129e65df2ec3 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.054149 May 19 12:38:34.123410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:34.123962 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1308/2741] 10.222.0.35 () {62 vars in 1199 bytes} [Thu May 19 12:38:34 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 56 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 19 12:38:34.473272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:34.484998 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7f3a1fe1-407f-48ee-8a5f-0a5e52016808 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.376582 May 19 12:38:34.486033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1434/2742] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:34.672178 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:34.675114 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-9272b631-184d-4180-9031-331ac92ff8a1 req-8d566001-2111-404f-940a-a761a5e366ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4518b411-8fac-48f2-8826-7b91651826b1", "name": "network-vif-deleted", "tag": "cf5a95fb-957c-4a6a-8f4e-61dcbbae636e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:34.677282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:34.690731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:34.695018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:34.696195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:34.697720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9272b631-184d-4180-9031-331ac92ff8a1 req-8d566001-2111-404f-940a-a761a5e366ee service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:34.698362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:34.703215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9272b631-184d-4180-9031-331ac92ff8a1 req-8d566001-2111-404f-940a-a761a5e366ee service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:34.725352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-9272b631-184d-4180-9031-331ac92ff8a1 req-8d566001-2111-404f-940a-a761a5e366ee service nova] Creating event network-vif-deleted:cf5a95fb-957c-4a6a-8f4e-61dcbbae636e for instance 4518b411-8fac-48f2-8826-7b91651826b1 on hv-cinder-80403 May 19 12:38:34.739858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-9272b631-184d-4180-9031-331ac92ff8a1 req-8d566001-2111-404f-940a-a761a5e366ee service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081707 May 19 12:38:34.740911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1435/2743] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:38:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:38:34.797444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:34.806911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42592a01-b0e8-49f8-982f-b6da31b570af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1650 microversion: 2.1 time: 0.148468 May 19 12:38:34.807463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1309/2744] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1650 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:35.483703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1955470606", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e6e51914-d344-4c2c-9d9a-e967c10c1ab5"}], "security_groups": [{"name": "tempest-securitygroup--1265018556"}], "key_name": "tempest-keypair-1748312208", "user_data": "}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:35.502683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:35.503281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:35.511057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:35.511496 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:35.512474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:35.513021 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:35.601050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] validate_networks() for [('e6e51914-d344-4c2c-9d9a-e967c10c1ab5', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:38:35.660594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:35.670177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fc2eb3a1-3e82-4112-980e-909323067702 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171488 May 19 12:38:35.671209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1310/2745] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:35.823253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:35.823858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:35.830051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:35.830452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:35.831175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:35.831601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:35.893121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:35.901262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3e7bf455-6611-4fe3-9d9e-8ddfa1b575e5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.081843 May 19 12:38:35.901805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1311/2746] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:36.048275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:38:36.048718 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:38:36.049220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:38:36.049616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:38:36.050017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:38:36.059897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.060335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:36.061180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.061609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:36.079216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:38:36.083932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Getting quotas for user b4113cb440764819996cfffec45e0b76 and project 50ab5e13c7394e5e91323bc7ed76c5f8. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:38:36.096310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:38:36.098756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:38:36.144616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR oslo.messaging._drivers.impl_rabbit [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [23d5f452-058d-4548-aac1-ac1e6c7941c5] AMQP server on 10.222.0.35:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 19 12:38:36.692058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:36.693099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:36.704125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.704892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:36.706144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.706856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:36.842450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:36.855101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3afccf6b-43b7-4df0-8c40-00f9a3869ba8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169220 May 19 12:38:36.855894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1312/2747] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:36.921539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:36.922515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:36.932452 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.933250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:36.934491 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:36.935397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:37.003069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:37.011184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4e72422a-7df6-4a48-867e-d2c4d7d81e0c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095650 May 19 12:38:37.011759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1313/2748] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:37.156261 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [23d5f452-058d-4548-aac1-ac1e6c7941c5] Reconnected to AMQP server on 10.222.0.35:5672 via [amqp] client with port 48308. May 19 12:38:37.163012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d5b803f-5337-4806-af11-da3523c0ab2c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.684072 May 19 12:38:37.163671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1436/2749] 10.222.0.35 () {62 vars in 1187 bytes} [Thu May 19 12:38:35 2022] POST /compute/v2.1/servers => generated 402 bytes in 1686 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 19 12:38:37.173902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:37.174513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:37.232760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:37.240282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:37.241897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:37.243025 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f5d4391-87eb-457f-8d07-a513fd6ad5f5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1282 microversion: 2.20 time: 0.073021 May 19 12:38:37.243568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1314/2750] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:37 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1282 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:37.876839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:37.877432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:37.884815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:37.885257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:37.886004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:37.886427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:38.015459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:38.024267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-550b5587-6cec-4689-a50d-eca37794c747 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.152207 May 19 12:38:38.024879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1437/2751] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:38.027677 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:38.028316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:38.036186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:38.036604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:38.037390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:38.037818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:38.101084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:38.104592 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:38.109521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa86e2bf-f904-4454-9a82-c2c9da3bc5e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085780 May 19 12:38:38.110085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1315/2752] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:38.261724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:38.262290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:38.269121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:38.269561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:38.270345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:38.270766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:38.351059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:38.359909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:38.362247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:38.363580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0a4fbe7-5045-4eea-85c3-3ae9d6d37d36 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1352 microversion: 2.20 time: 0.106546 May 19 12:38:38.364350 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1438/2753] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:38 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1352 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:39.040921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:39.041611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:39.051311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.051764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.052736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.053227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.128393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:39.130330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:39.138057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.138920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.140685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.142658 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.199326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:39.207459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f786c8b4-64df-4224-8214-9d0a631950f3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.170543 May 19 12:38:39.208036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1316/2754] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:39.226656 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:39.235394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8548f206-834f-42a4-b7dc-084ae3e1c055 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.112856 May 19 12:38:39.236230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1439/2755] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:39.383224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:39.384002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:39.390339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.390814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.391547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.391972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.526849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:39.541983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:39.543796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:39.545446 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1be032e6-90fe-4b4a-873a-e88075055c9c tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.165815 May 19 12:38:39.545834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1317/2756] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:39 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:39.782742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-296fb833-2d1e-448e-b6f6-6aa786407e11 req-2e046f92-bf5e-4af4-bd73-17f55b94b777 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8090e01e-64e2-475e-8239-501c89cdd287", "tag": "4abae3dc-89de-48f0-b622-2728436e6a54"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:39.791855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-296fb833-2d1e-448e-b6f6-6aa786407e11 req-2e046f92-bf5e-4af4-bd73-17f55b94b777 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:39.792600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-296fb833-2d1e-448e-b6f6-6aa786407e11 req-2e046f92-bf5e-4af4-bd73-17f55b94b777 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:39.809589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-296fb833-2d1e-448e-b6f6-6aa786407e11 req-2e046f92-bf5e-4af4-bd73-17f55b94b777 service nova] Creating event network-changed:4abae3dc-89de-48f0-b622-2728436e6a54 for instance 8090e01e-64e2-475e-8239-501c89cdd287 on hv-cinder-80403 May 19 12:38:39.814940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-296fb833-2d1e-448e-b6f6-6aa786407e11 req-2e046f92-bf5e-4af4-bd73-17f55b94b777 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035746 May 19 12:38:39.815768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1440/2757] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:38:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:38:40.225697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:40.226306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:40.233272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.233716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.234501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.234926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.255033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:40.255666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:40.262413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.262917 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.263615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.264040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.327236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:40.335518 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e722f5b1-41f2-41a1-9886-68fff9f179b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085982 May 19 12:38:40.336070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1441/2758] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:40.367157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:40.375574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a916878f-3bae-419f-aa6f-06f2358e81cb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154211 May 19 12:38:40.376182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1318/2759] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:40 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:40.566444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:40.567422 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:40.580744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.581426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.582682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:40.583395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:40.725487 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:40.733952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:40.735426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:40.736525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f39fac3b-60fc-49f7-9011-12b0871d139a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.176616 May 19 12:38:40.737149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1442/2760] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:40 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:41.353265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:41.353872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:41.360931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.361380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.362135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.362563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.392600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:41.393225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:41.400391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.400964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.401827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.402289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.428000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:41.436243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-948afb51-4e5b-40d8-bb63-8a36253c7c2f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086764 May 19 12:38:41.436845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1319/2761] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:41.624003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:41.632036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5795476e-c03c-454e-82bd-98e2f1f42a62 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.243251 May 19 12:38:41.632625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1443/2762] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:41.759022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:41.759814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:41.769279 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.769964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.771176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:41.771723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:41.904033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:41.912276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:41.913791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:41.914894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b33f0560-6d8a-41ca-9878-c31dc3a314ba tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.162926 May 19 12:38:41.915454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1320/2763] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:41 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:42.453499 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:42.454086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:42.461115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.461549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:42.462297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.462711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:42.529179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:42.537686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9de820b5-19f0-469e-bb33-e19c63a683b1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088084 May 19 12:38:42.538249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1444/2764] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:42.651789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:42.652479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:42.661211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.661791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:42.662774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.663313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:42.809732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:42.822175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-da1da974-979a-4a25-a6c9-91538df40506 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.175420 May 19 12:38:42.823026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1321/2765] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:42.936251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:42.937272 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:42.946172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.946610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:42.947359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:42.947783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:43.088522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:43.097756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:43.099592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:43.100731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-05138ef7-89c9-4d2a-a59e-e032409b0e87 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.170972 May 19 12:38:43.101511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1445/2766] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:42 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:43.555848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:43.556454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:43.563059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:43.563504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:43.564254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:43.564684 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:43.631046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:43.639342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2cf02cce-1d8a-4d3e-b4b9-0abcc7b5dcce tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087550 May 19 12:38:43.639904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1322/2767] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:43.843813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:43.844439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:43.851603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:43.851986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:43.852771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:43.853197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:43.988819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:43.997954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-560e04ae-2a5d-46ad-967b-9de77f27a187 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.158016 May 19 12:38:43.998541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1446/2768] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:44.129323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:44.129885 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:44.140267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:44.140888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:44.141988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:44.142521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:44.292910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:44.301310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:44.302752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:44.303940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5b92bf09-3b6d-45c2-8c25-f9e60ce48619 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.181243 May 19 12:38:44.304536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1323/2769] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:44 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:44.660360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:44.661424 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:44.668251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:44.668721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:44.669477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:44.669905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:44.739360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:44.748468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cf150270-09c1-4eea-be8a-1079ba52298b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095625 May 19 12:38:44.749086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1447/2770] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:45.018769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:45.019764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:45.027079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.027533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.028307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.028769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.172788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:45.181044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f851e6e1-7af9-4187-a4ae-05f0a4b5d989 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168655 May 19 12:38:45.181624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1324/2771] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:45.325050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:45.325822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:45.335172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.335754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.336774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.337337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.478212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:45.487095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:45.488701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:45.489838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6fa6cd72-1897-40d5-a177-9e5507de7019 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.170353 May 19 12:38:45.490455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1448/2772] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:45 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:45.586193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-a81d78d2-cee8-42a1-9a17-0d8d144eb084 req-f941d70b-f9d3-4668-977a-fbe2f42281dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8090e01e-64e2-475e-8239-501c89cdd287", "name": "network-vif-plugged", "status": "completed", "tag": "4abae3dc-89de-48f0-b622-2728436e6a54"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:45.594529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-a81d78d2-cee8-42a1-9a17-0d8d144eb084 req-f941d70b-f9d3-4668-977a-fbe2f42281dc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.594965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-a81d78d2-cee8-42a1-9a17-0d8d144eb084 req-f941d70b-f9d3-4668-977a-fbe2f42281dc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.606102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-a81d78d2-cee8-42a1-9a17-0d8d144eb084 req-f941d70b-f9d3-4668-977a-fbe2f42281dc service nova] Creating event network-vif-plugged:4abae3dc-89de-48f0-b622-2728436e6a54 for instance 8090e01e-64e2-475e-8239-501c89cdd287 on hv-cinder-80403 May 19 12:38:45.611047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-a81d78d2-cee8-42a1-9a17-0d8d144eb084 req-f941d70b-f9d3-4668-977a-fbe2f42281dc service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089366 May 19 12:38:45.611642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1325/2773] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:38:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:38:45.765946 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:45.766581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:45.772904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.773335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.774095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:45.774546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:45.846094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:45.856257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7cd06af4-4d30-41f1-83e8-4db70f67b67b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.094243 May 19 12:38:45.856846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1449/2774] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:46.203771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:46.204833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:46.217964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.218699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.220000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.220987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.380450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:46.389040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-89198c29-d636-4537-8db0-6a06c267ab8b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.193169 May 19 12:38:46.389650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1326/2775] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:46.511427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:46.512328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:46.515539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:46.516920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:46.525264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.525695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.526443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.526905 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.545393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:46.674973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:46.687485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:46.689911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:46.691567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45a79803-98a0-4249-b591-4356893f2d2a tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1408 microversion: 2.20 time: 0.186532 May 19 12:38:46.692381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1450/2776] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:46 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1408 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:46.873395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:46.874011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:46.880251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.880699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.881557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:46.881984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:46.944746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:46.952890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3b9a6238-bdf4-46c0-a862-b21fc441d53c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.083672 May 19 12:38:46.953468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1327/2777] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:47.409591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:47.410615 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:47.417683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.418124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.418877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.419291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.566830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:47.575407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-02bf1236-b0c6-41f0-bdbb-9c5d54c06774 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172267 May 19 12:38:47.575965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1451/2778] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:47 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:47.714437 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:47.715443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:47.719275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:47.727226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.727681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.728428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.728898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.863591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:47.872774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:47.874260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:47.875421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e5e0c5ad-f256-46c4-859f-9a21b659d491 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1598 microversion: 2.20 time: 0.167574 May 19 12:38:47.875990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1328/2779] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:38:47 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1598 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:38:47.892751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:47.893243 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:47.899527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.899968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.900742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.901165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.970349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:47.970924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:47.977619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.978062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.978818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:47.979240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:47.985454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-05cac7c4-cf2f-4acb-9a86-674482dfb0ad tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.096548 May 19 12:38:47.990750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1452/2780] 10.222.0.35 () {60 vars in 1315 bytes} [Thu May 19 12:38:47 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:38:48.067537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:48.075710 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4743d22f-23e8-47a2-959e-301ae2c2b715 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.109695 May 19 12:38:48.076893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1329/2781] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:47 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:48.595809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:48.596418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:48.602588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:48.603021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:48.603766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:48.604187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:48.738906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:48.747711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35fabcbf-f43b-4558-a6e2-8b82a11c2839 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.156164 May 19 12:38:48.748494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1453/2782] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:48.987706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-0c0b0c55-c0a0-462b-896b-3ffd8627834c req-e419bbe4-5f99-495a-8ede-b5582f3956f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8090e01e-64e2-475e-8239-501c89cdd287", "tag": "4abae3dc-89de-48f0-b622-2728436e6a54"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:38:48.996777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0c0b0c55-c0a0-462b-896b-3ffd8627834c req-e419bbe4-5f99-495a-8ede-b5582f3956f2 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:48.997230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0c0b0c55-c0a0-462b-896b-3ffd8627834c req-e419bbe4-5f99-495a-8ede-b5582f3956f2 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:49.008434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-0c0b0c55-c0a0-462b-896b-3ffd8627834c req-e419bbe4-5f99-495a-8ede-b5582f3956f2 service nova] Creating event network-changed:4abae3dc-89de-48f0-b622-2728436e6a54 for instance 8090e01e-64e2-475e-8239-501c89cdd287 on hv-cinder-80403 May 19 12:38:49.013470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-0c0b0c55-c0a0-462b-896b-3ffd8627834c req-e419bbe4-5f99-495a-8ede-b5582f3956f2 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083326 May 19 12:38:49.014055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1330/2783] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:38:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:38:49.093067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:49.094087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:49.100744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:49.101129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:49.101920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:49.102349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:49.187733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:49.196247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-963ce6ee-c724-457b-9381-07c0e8e8c450 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.107229 May 19 12:38:49.201757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1454/2784] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:49.765768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:49.766463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:49.773410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:49.773856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:49.774620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:49.775051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:49.916543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:49.925126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38bcf6e8-c59f-4cd4-9486-3a9af8c2396a tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163649 May 19 12:38:49.925713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1331/2785] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:50.213142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:50.213745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:50.220890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:50.221334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:50.222089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:50.222515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:50.286800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:50.300071 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4261a4c0-7b92-4d15-961d-f7e2d7247fcd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.090701 May 19 12:38:50.301390 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1455/2786] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:50.946226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:50.947167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:50.958234 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:50.958977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:50.959938 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:50.960368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:51.097065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:51.105378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-45c9e176-b1b9-47bd-b64a-8dd62a32e467 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165614 May 19 12:38:51.105955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1332/2787] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:51.321405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:51.321980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:51.329124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:51.329564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:51.330305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:51.330724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:51.395667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:51.404151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5d7d55bd-f6f5-46ab-a9c3-028a86dde1d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087718 May 19 12:38:51.404766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1456/2788] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:52.125427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:52.127244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:52.135607 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:52.136084 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:52.136881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:52.137312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:52.270121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:52.274251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:52.279262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6d73b814-38ba-438f-89a7-96e5f3f293bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.160191 May 19 12:38:52.279808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1333/2789] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:52.425590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:52.426533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:52.434032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:52.434471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:52.435220 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:52.435649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:52.498100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:52.506355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2a3ce266-7980-4e27-82a9-0cb24249970c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087295 May 19 12:38:52.506934 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1457/2790] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:53.303221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:53.303977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:53.312483 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:53.313088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:53.314031 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:53.314571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:53.454525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:53.462895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ee576a1c-0dbf-452f-ab37-db9f29ae7f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165023 May 19 12:38:53.463477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1334/2791] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:53 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:53.527035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:53.527786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:53.538455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:53.538903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:53.539660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:53.540088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:53.611687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:53.619863 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ff7bcb28-662c-438e-a20b-c5ce06f495c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.099160 May 19 12:38:53.620407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1458/2792] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:53.628290 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:38:53.628457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:38:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:38:54.479660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:54.480381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:54.487268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:54.487715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:54.488455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:54.488905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:54.634877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:54.637664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:54.638266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:54.643225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d664fbcd-4970-4f14-9896-0aae83edf54d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167405 May 19 12:38:54.643988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1335/2793] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:54.645033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:54.645481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:54.646231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:54.646720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:54.707281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:54.715471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-532a8d66-3294-4b23-ab80-50c69d5103f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.081653 May 19 12:38:54.716023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1459/2794] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:38:55.662036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:55.662746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:55.669458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:55.669928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:55.670671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:55.671090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:55.731957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:55.732534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:55.739242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:55.739702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:55.740547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:55.740947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:55.800601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:55.808318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:55.808760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9eb4ac4b-4a32-4d54-aed8-ec7989c9c3ed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.151490 May 19 12:38:55.809422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1336/2795] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:55.816621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5b3120a-564d-4b6f-8174-77b9fe893602 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088593 May 19 12:38:55.817187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1460/2796] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:56.830188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:56.831264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:56.836812 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:56.837413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:56.839683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:56.840146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:56.841115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:56.841551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:56.844258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:56.844715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:56.846773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:56.846773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:56.908439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:56.916897 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1d02ee48-0171-431e-91b0-dbac12d0b238 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085779 May 19 12:38:56.917473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1461/2797] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:56.974566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:56.986484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a1c07d6e-17c0-40d2-bb2a-b3fa24ef0c25 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162660 May 19 12:38:56.987236 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1337/2798] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:57.935746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:57.936696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:57.946459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:57.946907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:57.947666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:57.948095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:58.006402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:58.007048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:58.014046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:58.014497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:58.015523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:58.015640 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:58.017717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:58.026020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-49baa290-a53a-4f17-98eb-498e76219d99 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095316 May 19 12:38:58.026596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1462/2799] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:57 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:58.158407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:58.168000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fceb2564-6d8a-4e1b-8557-94fc4bed3584 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166305 May 19 12:38:58.168573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1338/2800] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:59.046880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:59.047892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:59.056590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:59.057072 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:59.057820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:59.058254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:59.120238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:59.129213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-84c64f74-4488-459d-9083-4b4aa4da41d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088803 May 19 12:38:59.129791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1463/2801] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:38:59.185368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:38:59.185943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:38:59.192860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:59.193300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:59.194053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:38:59.194453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:38:59.329139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:38:59.337356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3d946e5a-97c2-4689-90ee-5b3d2a9b8a17 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155877 May 19 12:38:59.337927 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1339/2802] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:38:59 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:00.150604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:00.151716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:00.163271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:00.163690 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:00.164446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:00.164927 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:00.229980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:00.238302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-355530d8-6f32-46d4-9395-3533afa4e4c9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093946 May 19 12:39:00.239108 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1464/2803] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:00.358707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:00.359699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:00.370688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:00.371778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:00.373075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:00.373789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:00.514905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:00.525618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4349a4fb-1235-4294-a801-27d26802bae5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173435 May 19 12:39:00.526190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1340/2804] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:01.258851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:01.259856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:01.270744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:01.271646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:01.272392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:01.272849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:01.335228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:01.343648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ead152b6-f55e-480b-a094-9637b7fe42a2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.091302 May 19 12:39:01.344229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1465/2805] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:01.542507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:01.543145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:01.545468 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:01.550930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:01.551378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:01.552133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:01.552560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:01.687386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:01.695604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e812cc29-f892-4b0b-96e2-7c27529c322d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.157095 May 19 12:39:01.696220 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1341/2806] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:02.360380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:02.361039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:02.368115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:02.368551 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:02.369330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:02.369752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:02.437092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:02.445477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-12df4724-e3c6-4ad4-a3e3-febb8cd6256b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089021 May 19 12:39:02.446041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1466/2807] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:02.714842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:02.715530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:02.725725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:02.726170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:02.726929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:02.727366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:02.888301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:02.898570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-83678bc5-6e8e-4bbe-b166-9c0e5edd4f94 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.189534 May 19 12:39:02.899152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1342/2808] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:03.462670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:03.463301 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:03.465714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=84,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:03.471392 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:03.471828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:03.472582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:03.473037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:03.493498 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:03.537814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:03.545902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-393cc847-3257-491b-94b3-ba035b23ebf4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087221 May 19 12:39:03.546517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1467/2809] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:03.919415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:03.920349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:03.931244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:03.931987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:03.933311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:03.934016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:04.070233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:04.078771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b7f4d8ca-0b3e-4c19-856a-fd6a45fcef28 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165785 May 19 12:39:04.079342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1343/2810] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:04.564385 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:04.565219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:04.574089 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:04.574678 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:04.575666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:04.576237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:04.893745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:04.901872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1cee477a-2b1d-4294-8863-050a2a5baa7f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.342689 May 19 12:39:04.902411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1468/2811] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:05.099661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:05.100610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:05.111418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:05.112149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:05.113435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:05.114137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:05.251542 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:05.259899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-950c7787-7c6e-4c8d-8edd-5c65646281e4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166601 May 19 12:39:05.260464 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1344/2812] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:05.921731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:05.923254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:05.930937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:05.931384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:05.932154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:05.932579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:06.001709 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:06.012040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d218431c-3acb-4b53-806d-ea009fb98bf7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.096103 May 19 12:39:06.012966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1469/2813] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:06.277706 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:06.278287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:06.285332 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:06.285772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:06.286528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:06.287043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:06.439924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:06.450796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b10b0382-9ed5-4f8a-9ec3-1e03c8c5eaa7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.177211 May 19 12:39:06.451363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1345/2814] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:06 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:07.034855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:07.035822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:07.043464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:07.043901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:07.044673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:07.045101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:07.110226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:07.118405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a78a43f5-1570-4cb3-978a-fb7ff8090606 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089979 May 19 12:39:07.118971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1470/2815] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:07.475331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:07.476271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:07.487207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:07.487940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:07.489216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:07.489920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:07.630545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:07.638546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f754954-bc05-43a8-a8b1-f724471a079c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169845 May 19 12:39:07.639108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1346/2816] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:08.139661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:08.140717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:08.149741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:08.150183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:08.150920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:08.151341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:08.222246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:08.230672 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee020167-8ba2-4f32-95a8-f8fdaac4a510 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.097830 May 19 12:39:08.231264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1471/2817] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:08 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:08.659540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:08.660443 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:08.668242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:08.668671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:08.669442 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:08.669868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:08.812945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:08.825449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-21f06de6-82d1-46a1-935d-6cda12eb1078 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172354 May 19 12:39:08.826211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1347/2818] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:09.251822 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:09.252904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:09.260752 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:09.261190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:09.261929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:09.262352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:09.327130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:09.335241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-925e5fec-9837-4391-b4e3-55bb992abf3c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.090261 May 19 12:39:09.335803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1472/2819] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:09.847032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:09.848004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:09.855866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:09.856326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:09.857105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:09.857525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:09.991317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:10.003351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f838941-f0c3-4ed2-b9b7-cb998357a0f9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162436 May 19 12:39:10.004097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1348/2820] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:10.356630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:10.357747 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:10.368191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:10.368628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:10.369411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:10.369835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:10.435488 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:10.443630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-50da7b0e-ed18-4c21-8319-2fa0ec94d6af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093563 May 19 12:39:10.444196 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1473/2821] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:11.025644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:11.026444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:11.035253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:11.035836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:11.036868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:11.037463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:11.359800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:11.368493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-23860953-0588-48c5-aa93-a82d1e86cddf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.349088 May 19 12:39:11.369104 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1349/2822] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:11.470000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:11.471050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:11.480060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:11.480510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:11.481286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:11.481711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:11.544517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:11.552668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28972cd9-bc3a-4411-b2d6-e9e99b09af33 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089018 May 19 12:39:11.553419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1474/2823] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:12.390929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:12.391594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:12.399223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:12.399717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:12.400566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:12.401081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:12.420495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:12.550346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:12.559908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07d459e6-3247-4c50-9c37-199fbd92c5d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173344 May 19 12:39:12.560456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1350/2824] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:12 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:12.570336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:12.570948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:12.577654 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:12.578103 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:12.578857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:12.579291 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:12.645692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:12.654246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c662d003-e89c-4033-a815-52143fd5c218 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087783 May 19 12:39:12.654825 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1475/2825] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:13.582095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:13.583134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:13.594286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:13.595029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:13.596270 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:13.597054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:13.671330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:13.671982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:13.678892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:13.679333 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:13.680101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:13.680524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:13.726701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:13.734806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d1150544-f04c-4420-a42f-ea65f5f608db tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159207 May 19 12:39:13.735385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1351/2826] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:13.745623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:13.754176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-59247597-cb1e-4b82-a286-f99e24b3b82d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086816 May 19 12:39:13.754736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1476/2827] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:14.755085 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:14.756102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:14.765974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:14.766731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:14.767900 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:14.768575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:14.773983 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:14.774545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:14.781425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:14.781866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:14.782610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:14.783028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:14.843684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:14.852612 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-18d5a486-09c5-4558-a199-d8353f7963f9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.084491 May 19 12:39:14.853202 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1477/2828] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:15.141832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:15.154240 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a4d855b-9420-41e0-b00e-d34c32f89be3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.404870 May 19 12:39:15.155019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1352/2829] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:15.874021 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:15.874961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:15.884889 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:15.885331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:15.886077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:15.886524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:15.947865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:15.955928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f7bad8d5-f397-4d6f-8c43-668de505723f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089325 May 19 12:39:15.956495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1478/2830] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:16.179384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:16.180072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:16.187163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:16.187614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:16.188365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:16.188835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:16.326950 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:16.338123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-945f428e-4c8d-4d80-be1e-558a3d966b63 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165350 May 19 12:39:16.338682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1353/2831] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:16.977887 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:16.978840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:16.990001 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:16.990738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:16.992213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:16.992662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:17.066455 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:17.074904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7aecb9ea-c195-4db9-ac62-20f8637088f4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.104051 May 19 12:39:17.075493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1479/2832] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:17.364624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:17.365721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:17.377121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:17.377866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:17.379118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:17.379848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:17.521111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:17.529209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7f3d8906-b012-43a4-b936-7359a8098a50 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171110 May 19 12:39:17.529782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1354/2833] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:18.096049 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:18.097093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:18.105749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:18.106193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:18.106979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:18.107408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:18.176757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:18.185313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-419fff9a-f9c4-49dc-987f-65c5c7c21071 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095756 May 19 12:39:18.185882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1480/2834] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:18 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:18.549971 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:18.550591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:18.557292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:18.557742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:18.558492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:18.558921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:18.707782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:18.717221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-066f9ae4-a345-4ba0-b549-9a732759cd36 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171093 May 19 12:39:18.717786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1355/2835] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:19.206596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:19.207589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:19.217810 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:19.218342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:19.219454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:19.220025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:19.291283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:19.299768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3136cc32-e129-46be-bc5b-93741f8e67f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.099832 May 19 12:39:19.300336 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1481/2836] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:19.738423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:19.739394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:19.750868 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:19.751597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:19.752865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:19.753586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:19.887046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:19.899345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3b98f632-42b9-4985-8245-d1911138c003 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166954 May 19 12:39:19.900179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1356/2837] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:19 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:20.323172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:20.324183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:20.331550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:20.332011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:20.332823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:20.333267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:20.400436 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:20.408971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3bd4f925-d871-4fa1-abc7-80d8245244af tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.092354 May 19 12:39:20.409544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1482/2838] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:20.920800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:20.921801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:20.933117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:20.933959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:20.934723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:20.935151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:21.089878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:21.098313 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-afb60252-1af5-48ef-ad7d-254cb25e4caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.184139 May 19 12:39:21.098888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1357/2839] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:21.356723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:21.357684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:21.368910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:21.369639 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:21.371397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:21.371820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:21.395962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1bee80ba-1102-481e-8ce2-bf772d672948 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287/ips" status: 200 len: 150 microversion: 2.20 time: 0.046149 May 19 12:39:21.396501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1483/2840] 10.222.0.35 () {60 vars in 1288 bytes} [Thu May 19 12:39:21 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287/ips => generated 150 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 19 12:39:21.425942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:21.426519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:21.433138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:21.433620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:21.434382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:21.434784 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:21.502190 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:21.510839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-af4c7c5e-dd89-4074-bb5b-93274f334298 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088862 May 19 12:39:21.511404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1358/2841] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:22.119741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:22.121206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:22.128453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:22.128930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:22.129684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:22.130115 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:22.148891 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:22.272892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:22.281324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9cbdf2da-8446-4bcf-8f73-d295b25484b1 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168257 May 19 12:39:22.281859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1484/2842] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:22.533444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:22.534050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:22.541449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:22.541903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:22.542699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:22.543108 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:22.605729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:22.614820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0f4291ef-0c27-47c3-af86-a28466b2e43c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085370 May 19 12:39:22.615382 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1359/2843] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:23.300673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:23.301325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:23.308493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:23.308956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:23.309742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:23.310165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:23.448386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:23.458689 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5f8759de-32ba-4ce3-b829-5d81618b69b9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162823 May 19 12:39:23.459241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1485/2844] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:23.636629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:23.637674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:23.648707 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:23.649486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:23.650736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:23.651460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:23.720380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:23.728761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-95b3acc9-14cb-4141-9c75-ab674f949b5a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.098567 May 19 12:39:23.729318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1360/2845] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:24.479339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:24.480267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:24.491730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:24.492461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:24.493755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:24.494463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:24.634056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:24.642252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d89a37c0-1ab0-4bb9-9f41-ad4ec599ddad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169562 May 19 12:39:24.642833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1486/2846] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:24.746317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:24.746921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:24.753860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:24.754298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:24.755054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:24.755470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:24.819722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:24.828174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d1c1ffa8-ab07-433c-a9c5-1561125f664d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086617 May 19 12:39:24.828759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1361/2847] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:25.664492 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:25.665258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:25.672050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:25.672491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:25.673347 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:25.673770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:25.820720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:25.834084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7e5b5630-ead1-4087-b9da-338f3daa1da2 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.176387 May 19 12:39:25.835002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1487/2848] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:25 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:25.847880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:25.849054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:25.856826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:25.857285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:25.858059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:25.858492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:25.928348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:25.936899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-121a636e-eae0-47ad-8e81-3a6b10f3ecda tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095132 May 19 12:39:25.937470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1362/2849] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:26.860714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:26.861732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:26.871031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:26.871466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:26.872219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:26.872657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:26.954343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:26.954947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:26.961465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:26.961901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:26.962673 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:26.963091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:27.004497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:27.012817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28cf4ab6-cc8a-421e-a9aa-202a9dcd6886 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.158540 May 19 12:39:27.013397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1488/2850] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:27.026493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:27.034589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f9f61127-522d-4c12-a85c-0c9ebf97590e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.084128 May 19 12:39:27.035107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1363/2851] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:28.032025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:28.033056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:28.042268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:28.042723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:28.043517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:28.043942 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:28.051297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:28.052045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:28.058709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:28.059188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:28.059937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:28.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:28.123655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:28.131889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0a385413-8c46-4633-bb53-07a6c3a5cae5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.084430 May 19 12:39:28.132482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1364/2852] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:28 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:28.182372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:28.191284 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c0f8b930-c3f7-421e-9a66-910156bc4f5d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165349 May 19 12:39:28.191865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1489/2853] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:29.155251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:29.156250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:29.167341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:29.168105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:29.169386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:29.170115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:29.208219 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:29.208833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:29.215582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:29.216025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:29.216803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:29.217226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:29.234727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:29.242770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-029ce268-5e75-4ebe-a117-a5bfdbb44532 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093999 May 19 12:39:29.243343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1365/2854] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:29.350257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:29.358978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6ef6c385-b0c4-47c5-ae59-427552437795 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154549 May 19 12:39:29.359562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1490/2855] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:30.263424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:30.264328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:30.275386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:30.276020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:30.277314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:30.278018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:30.346980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:30.355284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5d8ac5d4-9e24-4b44-be5c-f93acfae4203 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.098515 May 19 12:39:30.355847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1366/2856] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:30.376066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:30.376657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:30.383589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:30.384027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:30.384802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:30.385228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:30.531151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:30.540763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-feca3878-4695-48d1-8b8d-b3fd0262b1c4 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168438 May 19 12:39:30.541379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1491/2857] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:31.374654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:31.375396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:31.384142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:31.384690 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:31.385650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:31.386185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:31.391053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:31.453252 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:31.461485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-239ccf21-4fda-4e0d-b7d9-b373f0dc96b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.092108 May 19 12:39:31.462034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1367/2858] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:31.559952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:31.560971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:31.568353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:31.568873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:31.569776 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:31.570197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:31.718133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:31.726372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-991c96fd-0aef-4f66-9cc3-e2027d0f3b60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171333 May 19 12:39:31.727107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1492/2859] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:31 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:32.482421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:32.483414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:32.486922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=54 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:32.496887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:32.497653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:32.498846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:32.499549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:32.572369 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:32.584187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d862b33c-5bb6-4911-b814-679eb26e13be tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.108021 May 19 12:39:32.585005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1368/2860] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:32.748014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:32.748956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:32.760065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:32.760852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:32.762076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:32.762774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:32.903826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:32.912335 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-08210e09-a54e-49ec-b8aa-9e3bb681f371 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171664 May 19 12:39:32.912936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1493/2861] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:33.610676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:33.611487 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:33.621018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:33.621453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:33.622208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:33.622636 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:33.686376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:33.698047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-aaf23c2c-a1f6-4c4f-bea6-899f2984db75 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093580 May 19 12:39:33.698588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1369/2862] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:33 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:33.934236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:33.935123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:33.938753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=54 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:33.946807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:33.947216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:33.947997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:33.948479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:34.090850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:34.098879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-92d07685-8d51-4ee9-8f6f-f680a1180c0b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171244 May 19 12:39:34.099442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1494/2863] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:34.719810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:34.720799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:34.729675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:34.730152 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:34.730865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:34.731288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:34.792734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:34.801210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a68ac4d-481d-4d39-86d7-fd19e79e4abd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088284 May 19 12:39:34.801783 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1370/2864] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:34 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:35.120058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:35.121494 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:35.132819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:35.133751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:35.135165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:35.135997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:35.288562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:35.296659 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a938dbd7-8e44-4761-ac6b-fb92bae9e3a9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.183153 May 19 12:39:35.297235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1495/2865] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:35 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:35.822839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:35.823756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:35.834702 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:35.835417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:35.836160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:35.836585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:35.896546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:35.904534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6f7f4206-ece5-495e-b600-c7a4a7a75b06 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088316 May 19 12:39:35.905119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1371/2866] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:35 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:36.317824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:36.318846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:36.326200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:36.326641 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:36.327391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:36.327815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:36.473251 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:36.485574 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5f0eb08b-0abb-4f15-a8ad-46691de76183 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173983 May 19 12:39:36.486418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1496/2867] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:36 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:36.925297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:36.926228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:36.937280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:36.937984 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:36.939229 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:36.939951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:37.008224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:37.016403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a600495d-e026-4c3c-a694-f8844dd999d9 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.097682 May 19 12:39:37.017000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1372/2868] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:36 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:37.509211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:37.510043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:37.521050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:37.521824 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:37.523009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:37.523450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:37.658100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:37.666804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-35029721-8e0d-4416-8bb2-20eedc0a2b89 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164305 May 19 12:39:37.667373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1497/2869] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:37 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:38.037996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:38.038977 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:38.050083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:38.050822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:38.052098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:38.052934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:38.114986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:38.123763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-656a1f00-078b-4b95-9d6a-d0859ce7320e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.092263 May 19 12:39:38.124339 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1373/2870] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:38 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:38.688176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:38.689139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:38.698686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:38.699109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:38.699909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:38.700334 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:38.829037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:38.837713 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a88d08e9-f995-44d0-8c8c-97931f2bc802 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.156239 May 19 12:39:38.838289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1498/2871] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:38 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:39.145181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:39.146162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:39.157380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:39.158155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:39.159357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:39.160071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:39.253307 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:39.261756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-929f3930-4ce4-41dd-a977-6bdc10ac5b22 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.123021 May 19 12:39:39.262315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1374/2872] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:39 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:39.864152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:39.865154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:39.876854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:39.877595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:39.878834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:39.879554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:40.017835 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:40.029772 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d3685342-bdb6-4e7c-b383-539ea60eb94d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171985 May 19 12:39:40.030526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1499/2873] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:39 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:40.283258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:40.284162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:40.294905 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:40.295635 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:40.296901 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:40.297806 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:40.369974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:40.382884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-26f98b2f-01f5-49f3-afb8-877f23643b6f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.105927 May 19 12:39:40.383740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1375/2874] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:40 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:41.050878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:41.051814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:41.059605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:41.060075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:41.060859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:41.061286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:41.199712 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:41.211298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d887b78-9591-4251-ac05-5af75abf413e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166996 May 19 12:39:41.211870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1500/2875] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:41 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:41.408257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:41.409283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:41.417391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:41.417831 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:41.418583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:41.419011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:41.490509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:41.498983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-68b5cc7c-c2ba-444a-90a2-17b88816f9c1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.097074 May 19 12:39:41.499553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1376/2876] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:41 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:42.237674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:42.238637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:42.249855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:42.250583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:42.251815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:42.252519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:42.258550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:42.409809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:42.418277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e8822a6e-a67c-441b-b018-86454933d2ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.187083 May 19 12:39:42.418852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1501/2877] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:42 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:42.519518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:42.520505 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:42.528261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:42.528731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:42.529475 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:42.529902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:42.598998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:42.607310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d74d5b7e-b44a-492c-bc10-feb575f56afa tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.094400 May 19 12:39:42.607855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1377/2878] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:42 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:43.442295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:43.443245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:43.454181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:43.454909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:43.456134 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:43.456879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:43.593445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:43.601793 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-72f3460f-d5a3-4a5a-89da-1b241ec5bd32 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165962 May 19 12:39:43.602368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1502/2879] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:43 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:43.627634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:43.628593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:43.636541 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:43.637106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:43.637856 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:43.638294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:43.714709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:43.723091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-00493475-fc0b-4a74-85de-499d4a0e0834 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.101899 May 19 12:39:43.723663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1378/2880] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:43 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:44.628147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:44.629207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:44.637255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:44.637694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:44.638437 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:44.638859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:44.741423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:44.742027 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:44.749281 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:44.749724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:44.750485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:44.751024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:44.785215 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:44.793545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d1a696dc-e45e-4520-915a-007d9b84a1bd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171717 May 19 12:39:44.794142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1503/2881] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:44 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:44.813698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:44.826212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab95f986-dff9-4db3-aead-0b8541f99991 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.088854 May 19 12:39:44.826929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1379/2882] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:44 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:45.814117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:45.815147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:45.823773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:45.824216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:45.825027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:45.825451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:45.846521 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:45.847130 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:45.853795 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:45.854243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:45.854999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:45.855448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:45.919394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:45.927676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e1d42394-8e2c-4ace-9102-703b7b7413d5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085012 May 19 12:39:45.928261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1380/2883] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:45 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:45.960289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:45.972577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ff2d3aa-064e-4b50-853d-6129306e7caf tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164605 May 19 12:39:45.973448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1504/2884] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:45 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:46.948873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:46.949802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:46.960689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:46.961159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:46.961918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:46.962340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:46.990309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:46.990935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:46.997857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:46.998305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:46.999054 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:46.999476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:47.032397 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:47.041060 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0db7a153-9006-46f0-9388-4e7e3bd784a3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.098716 May 19 12:39:47.041983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1381/2885] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:46 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:47.141044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:47.149550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e0d7a21-8516-466a-9757-b157e33926d9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.162994 May 19 12:39:47.150328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1505/2886] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:46 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:39:48.060344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:48.061080 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:48.068956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:48.069565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:48.070310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:48.070735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:48.144488 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:48.156579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-809fff91-3a2b-4e80-bc4e-3e51d78c27cb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.100668 May 19 12:39:48.157370 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1382/2887] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:48 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:48.167126 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:48.167744 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:48.174640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:48.175080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:48.175828 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:48.176252 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:48.310511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:48.318733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0ad90933-f56e-4ceb-906c-978541afdecd tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155568 May 19 12:39:48.319316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1506/2888] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:48 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:49.175062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:49.175691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:49.182667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:49.183107 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:49.183853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:49.184275 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:49.247501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:49.253287 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:49.256163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0fd37a34-2f79-4c43-9c44-d3229a7c805d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.085003 May 19 12:39:49.256777 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1383/2889] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:49 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:49.335602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:49.336203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:49.343433 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:49.343850 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:49.344603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:49.345069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:49.543410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:49.551697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0293c6aa-6601-435c-aae1-e880fed47be5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.220227 May 19 12:39:49.552275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1507/2890] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:49 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:50.281064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:50.281980 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:50.293937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:50.294571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:50.295811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:50.296528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:50.364373 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:50.373095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a64193db-3ebf-4f5e-92eb-fd7866483c40 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.098743 May 19 12:39:50.373667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1384/2891] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:50 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:50.572687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:50.573554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:50.582439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:50.583029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:50.584029 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:50.584594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:50.734620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:50.742786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a9976be-1405-4d08-a9ee-f0d3f232cee3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.176496 May 19 12:39:50.743484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1508/2892] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:50 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:51.394260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:51.395194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:51.403975 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:51.404416 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:51.405199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:51.405618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:51.470538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:51.482841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-83fb0c31-279c-4810-ad8b-46a62df28102 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.094841 May 19 12:39:51.483622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1385/2893] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:51 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:51.765647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:51.766532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:51.777417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:51.778311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:51.779112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:51.779554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:51.919036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:51.930461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-071e9d9d-7751-49d2-9345-aa3271f1129f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.171829 May 19 12:39:51.931231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1509/2894] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:51 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:52.502053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:52.502728 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:52.510903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:52.511423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:52.512299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:52.512835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:52.575763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:52.583748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1436fdaa-208e-4520-9f40-9cf0e58407d2 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086338 May 19 12:39:52.584310 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1386/2895] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:52 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:52.954535 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:52.955235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:52.963629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:52.964070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:52.964852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:52.965273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:53.106501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:53.115095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b9eed6b-4025-468f-b69c-059ff0219c75 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.165411 May 19 12:39:53.115647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1510/2896] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:52 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:53.603940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:53.604556 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:53.611512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:53.611958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:53.612744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:53.613171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:53.677871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:53.686020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3f1beac8-8dcb-40d4-8889-83421ecd6534 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.087882 May 19 12:39:53.686581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1387/2897] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:53 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:54.136846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:54.137797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:54.148731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:54.149459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:54.150721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:54.151435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:54.303770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:54.315510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2f855df1-9947-4d20-97f5-b934babed7c7 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.184999 May 19 12:39:54.316188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1511/2898] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:54 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:54.703462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:54.704083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:54.710978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:54.711504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:54.712262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:54.712693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:54.786886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:54.795323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-84f45e4b-bedf-41a3-b028-32ec389cc951 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095849 May 19 12:39:54.795894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1388/2899] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:54 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:55.337274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:55.337916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:55.345117 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:55.345555 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:55.346381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:55.346807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:55.474780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:55.483440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ee71ce9c-1440-4683-a73b-73f14994ed97 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.151738 May 19 12:39:55.484022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1512/2900] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:55 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:55.814450 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:55.815210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:55.823623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:55.824063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:55.824847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:55.825277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:55.897826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:55.906066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1c8b8ff7-de55-4abf-baf2-85fb01ff3748 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.096909 May 19 12:39:55.906634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1389/2901] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:55 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:56.504703 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:56.505846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:56.513925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:56.514374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:56.515152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:56.515578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:56.658674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:56.667040 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-27c08947-48eb-4bb1-ba17-9455ee7e5b93 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.169144 May 19 12:39:56.667614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1513/2902] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:56 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:56.927449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:56.928395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:56.936051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:56.936497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:56.937357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:56.937787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:56.999933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:57.011574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-27934620-178d-400b-93c4-cb0bd89092b0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.090285 May 19 12:39:57.012432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1390/2903] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:56 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:57.688234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:57.689260 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:57.700008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:57.700789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:57.702027 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:57.702807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:57.847853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:57.856328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d10ef18d-1f88-44b8-9185-dd861d094731 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.174389 May 19 12:39:57.856952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1514/2904] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:57 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:58.032511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:58.033507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:58.042863 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:58.043117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:58.043866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:58.044286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:58.117643 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:58.130982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6be529c2-b1d9-4f01-8767-1306678cbf8a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.104555 May 19 12:39:58.131804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1391/2905] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:58 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:58.878829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:58.879702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:58.887953 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:58.888391 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:58.889165 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:58.889579 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:59.025783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:59.038856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c208e121-abe1-47ed-a3a1-ace0cdf30b99 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166567 May 19 12:39:59.039715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1515/2906] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:58 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:39:59.148829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:39:59.149403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:39:59.156261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:59.156733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:59.157479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:39:59.157903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:39:59.229766 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:39:59.238098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6fab1bd9-42d8-4027-bbf7-ea90b3f9d2e3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093326 May 19 12:39:59.238669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1392/2907] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:39:59 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:00.060446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:00.061512 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:00.071956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:00.072394 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:00.073176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:00.073604 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:00.217091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:00.227715 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ebea3fbd-c296-4c38-a001-974eeae3c18f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173657 May 19 12:40:00.228287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1516/2908] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:00 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:00.255430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:00.255998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:00.263020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:00.263455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:00.264195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:00.264591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:00.333020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:00.341356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c06416db-6678-46f5-bf4f-1b78da43eb6b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089812 May 19 12:40:00.341926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1393/2909] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:00 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:01.247925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:01.248651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:01.256592 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:01.257184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:01.258078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:01.258552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:01.361952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:01.362579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:01.372081 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:01.373145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:01.374875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:01.375845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:01.406226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:01.409838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:01.414754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5b406de9-1fa7-4764-839d-abe5595ab74e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.172107 May 19 12:40:01.415371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1517/2910] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:01 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:01.443056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:01.451603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-39b9fb88-2302-43bd-a050-33fab9ea00a4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093548 May 19 12:40:01.452194 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1394/2911] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:01 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:02.435599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:02.436263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:02.443453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:02.444048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:02.444814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:02.445235 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:02.468912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:02.469494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:02.475634 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:02.476196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:02.476848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:02.477280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:02.548103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:02.557054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-77202ff5-f50f-485e-ba4c-1fce4b8d4266 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.091712 May 19 12:40:02.561383 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1395/2912] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:02 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:02.586991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:02.594971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d48d5eed-5e96-4af2-b09d-bd6578e8d3ac tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164635 May 19 12:40:02.595536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1518/2913] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:02 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:03.578127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:03.579120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:03.582719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=56 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:03.589819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:03.590264 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:03.591045 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:03.591474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:03.611521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:03.612197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:03.619147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:03.619557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:03.620671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:03.621127 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:03.658414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:03.665425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-74abdccb-dd08-4e56-91f4-565b51446a9b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093775 May 19 12:40:03.666051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1396/2914] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:03 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:03.754660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:03.763459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0f514049-87c0-49cf-ac63-18eb513d2142 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155814 May 19 12:40:03.763962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1519/2915] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:03 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:04.682782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:04.683381 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:04.690919 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:04.691385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:04.692238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:04.692754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:04.754757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:04.763458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9db187c0-849b-4d24-9897-aa56ffe2fe0e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.084818 May 19 12:40:04.763964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1397/2916] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:04 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:04.780679 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:04.781282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:04.783628 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=52 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:04.789139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:04.789606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:04.790447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:04.790874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:04.938605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:04.951086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-44df9b75-d79e-43f2-9315-975c64b40302 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.173992 May 19 12:40:04.951813 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1520/2917] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:04 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:05.784727 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:05.785615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:05.796705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:05.797485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:05.798782 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:05.799501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:05.862940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:05.872657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3daa8419-9302-4bf6-a3e3-0785b0902b4f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.094505 May 19 12:40:05.873245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1398/2918] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:05 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:05.972230 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:05.973193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:05.984282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:05.985099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:05.986338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:05.987070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:06.124263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:06.132379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-959dd7a7-8345-442c-8fa4-4c41c0a53ca3 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166639 May 19 12:40:06.132943 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1521/2919] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:05 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:06.892057 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:06.892731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:06.899679 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:06.900124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:06.900923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:06.901448 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:06.970036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:06.980614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d658d6a3-049f-4db0-8876-1a22ad4d6c5e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093353 May 19 12:40:06.981248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1399/2920] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:06 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:07.150673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:07.151616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:07.158443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:07.158884 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:07.159647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:07.160066 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:07.304925 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:07.313768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e06e12c-bbd4-4bf1-809f-e06ab2862a60 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167052 May 19 12:40:07.314269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1522/2921] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:07 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:08.001730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:08.002490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:08.012533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:08.013291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:08.014517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:08.015217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:08.036292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=100 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:08.082767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:08.091000 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b3ad8107-27cb-4d0f-a8b4-e8e147a3850e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.094462 May 19 12:40:08.091566 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1400/2922] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:07 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:08.335046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:08.336019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:08.346611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:08.347353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:08.348590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:08.349353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:08.485406 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:08.496978 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6ac6f8e3-0241-429e-9642-ece1e8953b01 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.168509 May 19 12:40:08.497737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1523/2923] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:08 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:09.111803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:09.112789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:09.124098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:09.124903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:09.126154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:09.126865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:09.193999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:09.202494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dd915cbc-809d-480d-8329-ead3823321b8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.097124 May 19 12:40:09.203054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1401/2924] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:09 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:09.519779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:09.520873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:09.533321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:09.533768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:09.534527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:09.534959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:09.678561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:09.691159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eecbde6d-f1ce-48c2-9645-3a3fd21156ea tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.177861 May 19 12:40:09.691742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1524/2925] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:09 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:10.223457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:10.224455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:10.235511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:10.236257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:10.237596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:10.238317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:10.302833 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:10.310929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-95e6fc37-ad5d-4606-95b0-ab3a023d01b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093958 May 19 12:40:10.311506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1402/2926] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:10 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:10.712495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:10.713530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:10.724594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:10.725381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:10.726595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:10.727447 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:10.861817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:10.870190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-adc86acd-56c3-45d2-b1ce-171752d5b227 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163722 May 19 12:40:10.870742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1525/2927] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:10 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:11.332110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:11.333093 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:11.341711 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:11.342079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:11.342826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:11.343248 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:11.406661 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:11.414741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e150665e-9136-4233-ade5-df5c4be484b7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089238 May 19 12:40:11.415301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1403/2928] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:11 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:11.891315 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:11.892244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:11.903031 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:11.903765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:11.905068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:11.905781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:12.044379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:12.052573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-868f1c6e-489a-437e-88c2-275f6669bcfb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.167736 May 19 12:40:12.053143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1526/2929] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:11 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:12.440568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:12.441576 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:12.450127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:12.450574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:12.451321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:12.451742 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:12.523511 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:12.531899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-eac60bc1-1e63-4d84-8221-a835f151eec5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.097794 May 19 12:40:12.532490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1404/2930] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:12 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:13.074496 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:13.075448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:13.086476 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:13.087069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:13.087829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:13.088257 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:13.238310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:13.249613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-116f9554-e5fa-4aa5-8718-5f3134f9fa5b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.181719 May 19 12:40:13.250432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1527/2931] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:13 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:13.553122 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:13.554132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:13.561540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:13.562002 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:13.562748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:13.563173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:13.634479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:13.647389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0b9f5edc-efa9-4a58-8e50-1a608f939de3 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.100427 May 19 12:40:13.648136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1405/2932] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:13 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:14.273544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:14.274531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:14.285613 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:14.286349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:14.287594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:14.288305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:14.424932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:14.433380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-850b1ac0-cfda-42d5-914d-f8136629be3e tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.166279 May 19 12:40:14.433965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1528/2933] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:14 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:14.667415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:14.667961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:14.674779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:14.675217 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:14.675956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:14.676378 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:14.747850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:14.755904 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4be27130-d781-40c0-987f-21036f721725 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093683 May 19 12:40:14.756476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1406/2934] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:14 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:15.454169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:15.455100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:15.466223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:15.466958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:15.468183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:15.468924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:15.615860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:15.624755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aeb6794f-f794-4a8a-b27a-b4437f1060fa tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.176996 May 19 12:40:15.625328 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1529/2935] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:15 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:15.778473 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:15.779097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:15.786180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:15.786632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:15.787375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:15.787796 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:15.850597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:15.858822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19ee9f29-b0af-4633-9f80-da5349da45fb tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.084512 May 19 12:40:15.859398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1407/2936] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:15 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:16.646160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:16.647135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:16.655364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:16.655804 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:16.656554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:16.657010 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:16.786218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:16.794788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45113c57-0534-4886-a356-da5bb51c477c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.155120 May 19 12:40:16.795363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1530/2937] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:16 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:16.880162 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:16.881144 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:16.889326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:16.889768 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:16.890517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:16.890941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:16.955034 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:16.962986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ab27a29-b8fa-4a28-8270-28ebd7eb859e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.089265 May 19 12:40:16.963555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1408/2938] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:16 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:17.814161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:17.815004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:17.824292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:17.824777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:17.825534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:17.825928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:17.961245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:17.973181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-67b1ee77-b702-43c2-a1a3-235a30643a4b tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.164564 May 19 12:40:17.973745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1531/2939] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:17 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:17.979302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:17.979933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:17.986771 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:17.987208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:17.987945 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:17.988365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:18.050621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:18.059016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-92c3d181-687f-4f7f-834b-eaa5d5276435 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.083491 May 19 12:40:18.059457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1409/2940] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:17 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:18.994166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:18.995091 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:19.002015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:19.002454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:19.003210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:19.003636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:19.075807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:19.076414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:19.082764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:19.083210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:19.083959 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:19.084389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:19.143416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:19.146243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:19.151671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d46a4459-f8c7-481e-a2b3-9fa39d79f7ad tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163788 May 19 12:40:19.152283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1532/2941] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:18 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:19.154507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4de00485-f095-49ed-a9dd-89da7e1af737 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.082785 May 19 12:40:19.155036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1410/2942] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:19 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:40:20.172846 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:20.173831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:20.174708 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:20.175724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:20.184132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:20.184911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:20.186143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:20.186841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:20.187096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:20.187747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:20.189094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:20.189818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:20.255230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:20.263859 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5ce3f8c7-757b-40bc-acdf-5f5eb5d7794f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095546 May 19 12:40:20.264561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1411/2943] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:20 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:20.322023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:20.330300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ed9c7415-ed85-4fd0-b88b-cffb703d5bf0 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163997 May 19 12:40:20.330866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1533/2944] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:20 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:21.285250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:21.286185 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:21.297291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.298030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.299271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.299994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.347460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:21.348074 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:21.354892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.355327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.356199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.356622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.365965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:21.374267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-66c6e81a-bdc8-4e7d-aa2a-f7d8000127e8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.095471 May 19 12:40:21.375046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1412/2945] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:21 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:21.375892 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=100 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:21.469748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:21.470241 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:21.476656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.477116 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.477880 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.478347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.488834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:21.497730 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4b083d5a-e6e1-4c63-b289-fdb2ee68abbe tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.154147 May 19 12:40:21.498474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1534/2946] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:21 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:21.503593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:40:21.598681 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8b0c6aeb-f1f8-43b5-bb6b-dc73f0e86c79 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 204 len: 0 microversion: 2.20 time: 0.132656 May 19 12:40:21.599348 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1413/2947] 10.222.0.35 () {60 vars in 1279 bytes} [Thu May 19 12:40:21 2022] DELETE /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 19 12:40:21.609166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:21.610077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:21.619197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.620526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.620945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:21.621578 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:21.753517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:21.761910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:21.763398 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:21.764548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5571b2f0-a9c3-4fe2-98ab-130bc36ab5c5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1723 microversion: 2.20 time: 0.160764 May 19 12:40:21.765139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1535/2948] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:21 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:40:22.395596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:22.396749 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:22.407423 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.408170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.409458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.410176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.490573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:22.498982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-79cb28c8-16b2-4c07-82e4-0ea576c70253 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.110420 May 19 12:40:22.499553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1414/2949] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:22 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:22.515534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:22.516147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:22.523167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.523618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.524373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.524848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.666330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:22.674854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5831c2e6-0e66-43c1-b316-14f69585b13c tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.163302 May 19 12:40:22.675396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1536/2950] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:22 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:22.787174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:22.788211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:22.798546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.799299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.800539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:22.801308 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:22.821832 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:22.936344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:22.944250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:22.945773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:22.946934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9a27664c-0012-4ea0-ac93-6d00187f89c9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1723 microversion: 2.20 time: 0.166515 May 19 12:40:22.947544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1415/2951] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:22 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:40:23.516069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:23.516663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:23.525657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.526197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:23.528367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.528367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:23.619415 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:23.628051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2ad4895d-30ac-43c4-8dc6-d0b3575bc43b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.116011 May 19 12:40:23.628745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1537/2952] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:23 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:23.693298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:23.694935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:23.701985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.702425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:23.703255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.703685 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:23.838269 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:23.846225 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-385343d2-1ae4-4cb5-a00b-7fd384c1d7fb tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.158752 May 19 12:40:23.846797 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1416/2953] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:23 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:23.965175 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:23.966183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:23.972891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.973413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:23.974224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:23.974858 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:24.053694 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:24.068838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-a2621a0f-4716-40a4-bcd4-b81ee6dc98cd req-bf6418d4-3698-42db-8370-1a80aa584eea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8090e01e-64e2-475e-8239-501c89cdd287", "name": "network-vif-deleted", "tag": "4abae3dc-89de-48f0-b622-2728436e6a54"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:24.077931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:24.080024 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:24.081527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6c905fb7-293b-4745-9434-a6780724c902 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 200 len: 1655 microversion: 2.20 time: 0.120250 May 19 12:40:24.092998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1417/2954] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:23 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 1655 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 19 12:40:24.096532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-a2621a0f-4716-40a4-bcd4-b81ee6dc98cd req-bf6418d4-3698-42db-8370-1a80aa584eea service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:24.097280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-a2621a0f-4716-40a4-bcd4-b81ee6dc98cd req-bf6418d4-3698-42db-8370-1a80aa584eea service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:24.110695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-a2621a0f-4716-40a4-bcd4-b81ee6dc98cd req-bf6418d4-3698-42db-8370-1a80aa584eea service nova] Creating event network-vif-deleted:4abae3dc-89de-48f0-b622-2728436e6a54 for instance 8090e01e-64e2-475e-8239-501c89cdd287 on hv-cinder-80403 May 19 12:40:24.115882 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-a2621a0f-4716-40a4-bcd4-b81ee6dc98cd req-bf6418d4-3698-42db-8370-1a80aa584eea service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.163805 May 19 12:40:24.116567 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1538/2955] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:24.645565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:24.646271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:24.652994 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:24.653551 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:24.654454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:24.654958 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:24.719115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:24.727818 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-20222255-a966-4db9-a15a-17bdf9a9c983 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086228 May 19 12:40:24.737195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1418/2956] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:24 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:24.864670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:24.865418 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:24.872271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:24.872740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:24.873491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:24.873918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.010774 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:25.019214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1ad3d422-58e1-4d54-a331-310127d06c34 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.159295 May 19 12:40:25.019692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1539/2957] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:24 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:25.104030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:25.105091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:25.115417 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.116157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.117432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.118141 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.132957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] HTTP exception thrown: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Request method failure captured: May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 HTTP/1.1 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKWKa77GOCoWGZnfFTh92SXUptKZJNjMQAIAscMUWYumhn4JKXiCveqs4IrToCF1ZsDRCS3ww34ohsw7nLc6F_uZk9NXmz9QQgihLXzfbY1BrOpBob-kCcmLFgbtvOV4mb9rinC3VipIPl82nlVp1_QXOWXmvUw0ft4oNOmytNfXqc May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Openstack-Nova-Api-Version: 2.20 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:25.134678 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}] May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b4113cb440764819996cfffec45e0b76 May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.136509 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '8090e01e-64e2-475e-8239-501c89cdd287'} May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:25.137663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.140061 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.141898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Returning 404 to user: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:25.141898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8f2859e8-0983-4a7c-bf71-5286a49fc0e5 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 404 len: 111 microversion: 2.20 time: 0.038496 May 19 12:40:25.141898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1419/2958] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:25 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 19 12:40:25.146664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cbac16ef-b41f-4199-a200-fe2ebf750444 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '.version_select at 0x7fe21134d430>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:25.166655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cbac16ef-b41f-4199-a200-fe2ebf750444 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1748312208" status: 204 len: 0 microversion: 2.20 time: 0.023820 May 19 12:40:25.167212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1540/2959] 10.222.0.35 () {60 vars in 1261 bytes} [Thu May 19 12:40:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1748312208 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 19 12:40:25.745149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:25.745876 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:25.753079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.753549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.754291 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.754735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.808556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:25.809063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:25.815168 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.815602 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.816345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.816798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.818952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:25.827221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3f5a6886-2893-478e-9737-4653681c8d51 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086433 May 19 12:40:25.827792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1420/2960] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:25 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:25.833530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] HTTP exception thrown: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Request method failure captured: May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 HTTP/1.1 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKWKa77GOCoWGZnfFTh92SXUptKZJNjMQAIAscMUWYumhn4JKXiCveqs4IrToCF1ZsDRCS3ww34ohsw7nLc6F_uZk9NXmz9QQgihLXzfbY1BrOpBob-kCcmLFgbtvOV4mb9rinC3VipIPl82nlVp1_QXOWXmvUw0ft4oNOmytNfXqc May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Openstack-Nova-Api-Version: 2.20 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:40:25.835590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}] May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: b4113cb440764819996cfffec45e0b76 May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.837596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '8090e01e-64e2-475e-8239-501c89cdd287'} May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:25.838607 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.840045 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:25.841841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Returning 404 to user: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:25.841841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-acb82774-f377-4104-8c76-1ba5dba5d252 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 404 len: 111 microversion: 2.20 time: 0.032561 May 19 12:40:25.841841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1541/2961] 10.222.0.35 () {60 vars in 1279 bytes} [Thu May 19 12:40:25 2022] DELETE /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 19 12:40:25.845303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:25.845929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: 8090e01e-64e2-475e-8239-501c89cdd287] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:25.852125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.852569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.853334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:25.853754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:25.866817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] HTTP exception thrown: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Request method failure captured: May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 HTTP/1.1 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKWKa77GOCoWGZnfFTh92SXUptKZJNjMQAIAscMUWYumhn4JKXiCveqs4IrToCF1ZsDRCS3ww34ohsw7nLc6F_uZk9NXmz9QQgihLXzfbY1BrOpBob-kCcmLFgbtvOV4mb9rinC3VipIPl82nlVp1_QXOWXmvUw0ft4oNOmytNfXqc May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Openstack-Nova-Api-Version: 2.20 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:25.868330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}] May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b4113cb440764819996cfffec45e0b76 May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.870726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '8090e01e-64e2-475e-8239-501c89cdd287'} May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:25.871886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. May 19 12:40:25.873671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:25.875209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Returning 404 to user: Instance 8090e01e-64e2-475e-8239-501c89cdd287 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:25.875209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fd720a31-5d3d-41b8-8cc3-e820e812bdc9 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287" status: 404 len: 111 microversion: 2.20 time: 0.028188 May 19 12:40:25.875209 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1421/2962] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:25 2022] GET /compute/v2.1/servers/8090e01e-64e2-475e-8239-501c89cdd287 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 19 12:40:26.041248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:26.041794 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:26.048650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:26.049106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:26.049849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:26.050265 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:26.179360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:26.187869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-61204014-d408-401f-90c7-9d736fbccc08 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.153171 May 19 12:40:26.188463 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1542/2963] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:26 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:26.848668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:26.849639 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:26.857966 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:26.858414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:26.859160 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:26.859587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:26.928347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:26.936751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-565b97c1-5891-4be2-8c09-1fcec83e4ced tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.093980 May 19 12:40:26.937299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1422/2964] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:26 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:27.212901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:27.213514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:27.220061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:27.220508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:27.221288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:27.221691 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:27.604306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:27.616277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9d7c6c76-e47f-4fcf-ad4d-538588202d73 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1737 microversion: 2.1 time: 0.409671 May 19 12:40:27.617249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1543/2965] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:27 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1737 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:27.954181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:27.954842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:27.961781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:27.962227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:27.963087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:27.963699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:28.028725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:28.037062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-dec14fb0-4d6e-4e00-bbf5-5ea60d05c45b tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1368 microversion: 2.1 time: 0.086914 May 19 12:40:28.037623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1423/2966] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:27 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:28.639749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:28.640506 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:28.647876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:28.648300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:28.649088 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:28.649509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:28.782076 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:28.789701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-45d364b5-4c67-4c35-bed7-a0f0521caaf8 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1691 microversion: 2.1 time: 0.154146 May 19 12:40:28.790263 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1544/2967] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:28 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1691 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:29.058226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:29.059182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:29.070918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.071656 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.072790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.073226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.110862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.111311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.111756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'fault' on Instance uuid 4518b411-8fac-48f2-8826-7b91651826b1 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:29.159407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:29.167528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-feb13969-3d85-4e7e-953c-b1c7b6551a74 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1462 microversion: 2.1 time: 0.115826 May 19 12:40:29.168067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1424/2968] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:29 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1462 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:29.379156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:29.379671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:29.385795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.386237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.386986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.387417 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.412223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:40:29.486363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eb79d2c2-bb5f-440f-8a09-4be6e79463bb tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 204 len: 0 microversion: 2.20 time: 0.110813 May 19 12:40:29.486918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1545/2969] 10.222.0.35 () {60 vars in 1279 bytes} [Thu May 19 12:40:29 2022] DELETE /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 19 12:40:29.495504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:29.496088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:29.502389 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.502816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.503569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.503988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.536575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.537040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.537480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lazy-loading 'fault' on Instance uuid b642ea31-4427-475b-8374-591f4dcbadc8 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:29.578724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:29.587367 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:29.588953 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:29.590102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-834beb5b-935e-4705-a13d-3005ad76fb6d tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1545 microversion: 2.20 time: 0.099016 May 19 12:40:29.590674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1425/2970] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:29 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1545 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:40:29.816353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:29.817546 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:29.826521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.826965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.827721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:29.828153 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:29.975452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:29.983547 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14ba07e4-9da0-4f51-aa97-ed963d54cf81 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1691 microversion: 2.1 time: 0.174302 May 19 12:40:29.984096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1546/2971] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:29 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1691 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:30.189943 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:30.190942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:30.201893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.202628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.203852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.204558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.238242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.238688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.239127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'fault' on Instance uuid 4518b411-8fac-48f2-8826-7b91651826b1 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:30.283424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:30.296032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-942aa840-8d61-44b6-ab95-81dab519f2e0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1462 microversion: 2.1 time: 0.113054 May 19 12:40:30.296915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1426/2972] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:30 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1462 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:30.701733 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:30.702339 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:30.708472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.708955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.709699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.710123 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.744964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:30.745397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:30.745839 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lazy-loading 'fault' on Instance uuid b642ea31-4427-475b-8374-591f4dcbadc8 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:30.787968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:30.796844 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:30.798354 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b4113cb440764819996cfffec45e0b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50ab5e13c7394e5e91323bc7ed76c5f8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:30.799548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0d2a0cce-2732-4a30-a947-cc31ec10dfd2 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 200 len: 1545 microversion: 2.20 time: 0.193401 May 19 12:40:30.800113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1547/2973] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:30 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 1545 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 19 12:40:31.004426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:31.005365 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:31.015553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.016298 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.017884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.018605 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.162747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:31.171949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-866b7bf1-70b1-4228-97e1-8a962b2bad56 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1691 microversion: 2.1 time: 0.174121 May 19 12:40:31.172524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1427/2974] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:30 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1691 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:31.321823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:31.322683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:31.330077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.330647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.331480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.332069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.364852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.365327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.365781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'fault' on Instance uuid 4518b411-8fac-48f2-8826-7b91651826b1 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:31.407129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:31.415360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fdff6eed-a1d0-4575-82a2-5e39d1a58239 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 200 len: 1462 microversion: 2.1 time: 0.100204 May 19 12:40:31.416120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1548/2975] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:31 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 1462 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:31.817092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:31.817689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] [instance: b642ea31-4427-475b-8374-591f4dcbadc8] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:31.824549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.825039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.825803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:31.826273 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:31.840326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] HTTP exception thrown: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Request method failure captured: May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 HTTP/1.1 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKWKa77GOCoWGZnfFTh92SXUptKZJNjMQAIAscMUWYumhn4JKXiCveqs4IrToCF1ZsDRCS3ww34ohsw7nLc6F_uZk9NXmz9QQgihLXzfbY1BrOpBob-kCcmLFgbtvOV4mb9rinC3VipIPl82nlVp1_QXOWXmvUw0ft4oNOmytNfXqc May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Openstack-Nova-Api-Version: 2.20 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:31.841932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_50ab5e13c7394e5e91323bc7ed76c5f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/50ab5e13c7394e5e91323bc7ed76c5f8"}]}] May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 50ab5e13c7394e5e91323bc7ed76c5f8 May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2047828675 May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b4113cb440764819996cfffec45e0b76 May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2047828675-project May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. May 19 12:40:31.843827 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'b642ea31-4427-475b-8374-591f4dcbadc8'} May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:31.845059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. May 19 12:40:31.846732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:31.848436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] Returning 404 to user: Instance b642ea31-4427-475b-8374-591f4dcbadc8 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:31.848436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e374aaf3-66af-4c6d-a218-c1b1f16764a0 tempest-AttachVolumeShelveTestJSON-2047828675 tempest-AttachVolumeShelveTestJSON-2047828675-project] 10.222.0.35 "GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8" status: 404 len: 111 microversion: 2.20 time: 0.030288 May 19 12:40:31.848436 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1428/2976] 10.222.0.35 () {60 vars in 1276 bytes} [Thu May 19 12:40:31 2022] GET /compute/v2.1/servers/b642ea31-4427-475b-8374-591f4dcbadc8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 19 12:40:32.189142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:32.189770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:32.195932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:32.196380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:32.197270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:32.197702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:32.437087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:32.438557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:32.446598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:32.447314 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:32.448318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:32.449204 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:32.467180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 HTTP/1.1 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:32.469537 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:40:32.471109 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '4518b411-8fac-48f2-8826-7b91651826b1'} May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:32.472138 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:40:32.473655 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:32.476200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:32.477311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1b788dcb-1a89-4546-b0b2-1038804cc574 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 404 len: 111 microversion: 2.1 time: 0.048078 May 19 12:40:32.482253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1429/2977] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 19 12:40:32.488058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-656d3390-5c03-4900-ac07-c7f0f386c36f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '.version_select at 0x7fe20c407160>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:32.547747 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-656d3390-5c03-4900-ac07-c7f0f386c36f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-973024615" status: 202 len: 0 microversion: 2.1 time: 0.064130 May 19 12:40:32.554518 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1430/2978] 10.222.0.35 () {58 vars in 1217 bytes} [Thu May 19 12:40:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-973024615 => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:40:32.600543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:32.616554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f59acabb-ccf6-4c81-aa77-e16981bf0f51 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1691 microversion: 2.1 time: 0.431200 May 19 12:40:32.621444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1549/2979] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:32 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1691 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:33.393889 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-fbd4a42a-9c61-49c6-92f6-a6df463c31b2 req-fb4b41ed-477b-4d39-9abf-4aff66217348 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e79bc9b-c29e-47f8-9d9d-2f99452ed121", "name": "network-vif-deleted", "tag": "f7066a86-57be-43fc-88d4-b9a677072655"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:33.403970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-fbd4a42a-9c61-49c6-92f6-a6df463c31b2 req-fb4b41ed-477b-4d39-9abf-4aff66217348 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:33.404410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-fbd4a42a-9c61-49c6-92f6-a6df463c31b2 req-fb4b41ed-477b-4d39-9abf-4aff66217348 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:33.420672 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-fbd4a42a-9c61-49c6-92f6-a6df463c31b2 req-fb4b41ed-477b-4d39-9abf-4aff66217348 service nova] Creating event network-vif-deleted:f7066a86-57be-43fc-88d4-b9a677072655 for instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 on hv-cinder-80403 May 19 12:40:33.426230 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-fbd4a42a-9c61-49c6-92f6-a6df463c31b2 req-fb4b41ed-477b-4d39-9abf-4aff66217348 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037387 May 19 12:40:33.431019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1431/2980] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:33.635986 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:33.637254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:33.651588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:33.651588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:33.658823 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:33.659409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:33.896397 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:33.909152 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2d697a7a-3684-4439-8132-04ab6ae000fc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 200 len: 1331 microversion: 2.1 time: 0.276255 May 19 12:40:33.910465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1550/2981] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:33 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 1331 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:34.112458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b037e8e1-af66-4ee7-8de0-10ea8af735d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: .version_select at 0x7fe1fff97af0>, body: {"keypair": {"name": "tempest-keypair-1293115722", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLf34hPNZ57xyJ6QMu3SoF/DCc3Q0fS8oyFvKYel0uaD+O/6ivtRYLGuspemIb7tymtC6kH7a3IKvfkpuhugBxH6/B53/taKwAQqaYV437Okn5c++GKTMNY85hwi/n5RfQ=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:34.121516 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:34.177422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-b037e8e1-af66-4ee7-8de0-10ea8af735d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:40:34.201465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-b037e8e1-af66-4ee7-8de0-10ea8af735d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for user eefde2311ec64430a6ed161238024df4 and project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:40:34.290641 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b037e8e1-af66-4ee7-8de0-10ea8af735d8 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.187061 May 19 12:40:34.291529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1432/2982] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:40:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:34.927996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:34.928683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: 0e79bc9b-c29e-47f8-9d9d-2f99452ed121] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:34.932528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=72,nova_api:DELETE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:34.937527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:34.937973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:34.939521 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:34.940326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:34.959781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:34.961867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 HTTP/1.1 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:40:34.963651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. May 19 12:40:34.965623 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '0e79bc9b-c29e-47f8-9d9d-2f99452ed121'} May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:34.966683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. May 19 12:40:34.968388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:34.970129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 404 to user: Instance 0e79bc9b-c29e-47f8-9d9d-2f99452ed121 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:34.970129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5d0b944a-2572-4c45-892a-e3ad6ee5ea4d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121" status: 404 len: 111 microversion: 2.1 time: 0.042518 May 19 12:40:34.970129 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1551/2983] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:34 2022] GET /compute/v2.1/servers/0e79bc9b-c29e-47f8-9d9d-2f99452ed121 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:34.977659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e2359ab9-6dc4-4144-bed0-9c8a04d7c658 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '.version_select at 0x7fe211331160>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:35.006265 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e2359ab9-6dc4-4144-bed0-9c8a04d7c658 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1886018830" status: 202 len: 0 microversion: 2.1 time: 0.032597 May 19 12:40:35.007659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1433/2984] 10.222.0.35 () {58 vars in 1220 bytes} [Thu May 19 12:40:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1886018830 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:40:35.447865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-f0d4121a-f9b6-4fc1-ba8c-7d37211834a6 req-7c2278e9-3bf2-40a0-b595-8aac50397425 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2382eba-ee5f-4026-8253-b5835cfb84c4", "name": "network-vif-deleted", "tag": "547bc25f-fbc2-430e-b777-1456f40c4c56"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:35.461630 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-f0d4121a-f9b6-4fc1-ba8c-7d37211834a6 req-7c2278e9-3bf2-40a0-b595-8aac50397425 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:35.462271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-f0d4121a-f9b6-4fc1-ba8c-7d37211834a6 req-7c2278e9-3bf2-40a0-b595-8aac50397425 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:35.479376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-f0d4121a-f9b6-4fc1-ba8c-7d37211834a6 req-7c2278e9-3bf2-40a0-b595-8aac50397425 service nova] Creating event network-vif-deleted:547bc25f-fbc2-430e-b777-1456f40c4c56 for instance e2382eba-ee5f-4026-8253-b5835cfb84c4 on hv-cinder-80403 May 19 12:40:35.488193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-f0d4121a-f9b6-4fc1-ba8c-7d37211834a6 req-7c2278e9-3bf2-40a0-b595-8aac50397425 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045259 May 19 12:40:35.489766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1552/2985] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:37.060819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-957097557", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "887eaf77-1004-4cec-8309-32f7b4c05c89"}], "security_groups": [{"name": "tempest-securitygroup--2106611685"}], "key_name": "tempest-keypair-1293115722", "user_data": "}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:37.238030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] validate_networks() for [('887eaf77-1004-4cec-8309-32f7b4c05c89', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:40:37.883716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:40:37.884133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:40:37.884667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:40:37.885125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:40:37.885527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:40:37.895405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:37.895846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:37.896658 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:37.897125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:37.917319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:40:37.921462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Getting quotas for user eefde2311ec64430a6ed161238024df4 and project c7917b551c3c4cc9be8a007b5cc987d3. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:40:37.935013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:40:37.937527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:40:37.992535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8baf8b8a-4ef0-4123-b7d2-f9b2638b8416 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.937200 May 19 12:40:37.998364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1434/2986] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:40:37 2022] POST /compute/v2.1/servers => generated 402 bytes in 939 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:40:38.004118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7e790f1c-7227-4f36-b042-e484891b467f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:38.004958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7e790f1c-7227-4f36-b042-e484891b467f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:38.061961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7e790f1c-7227-4f36-b042-e484891b467f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:38.069140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7e790f1c-7227-4f36-b042-e484891b467f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1237 microversion: 2.1 time: 0.069290 May 19 12:40:38.072680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1553/2987] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:37 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1237 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:38.219358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:38.220732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:38.230155 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:38.230847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:38.232321 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:38.233411 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:38.261215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:40:38.355226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-470e4b30-1ed9-42ff-9ddc-f787f881719f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 204 len: 0 microversion: 2.1 time: 0.141022 May 19 12:40:38.359601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1435/2988] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:40:38 2022] DELETE /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:40:38.366900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:38.367660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:38.377188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:38.378143 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:38.379014 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:38.379732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:38.520970 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:38.528568 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-28f01909-cd14-4d5c-b369-ccbbb39da8d5 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1508 microversion: 2.1 time: 0.166468 May 19 12:40:38.529248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1554/2989] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:38 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1508 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:38.932392 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-08020635-254d-45bd-9854-423c360f4058 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Action: 'create', calling method: .version_select at 0x7fe20c24f700>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-194001979"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:39.086566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:39.087169 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:39.094600 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:39.095082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:39.095932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:39.096543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:39.163697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:39.171356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0ae9313c-ffce-43e5-811a-59fbeef100d6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1241 microversion: 2.1 time: 0.088901 May 19 12:40:39.172034 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1555/2990] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:39 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1241 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:39.243015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-08020635-254d-45bd-9854-423c360f4058 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.432933 May 19 12:40:39.243651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1436/2991] 10.222.0.35 () {60 vars in 1154 bytes} [Thu May 19 12:40:38 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 434 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 19 12:40:39.255329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7da1cf6e-bec9-46ce-9c9e-adb036f38f3c tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe211333c10>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:39.303376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7da1cf6e-bec9-46ce-9c9e-adb036f38f3c tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 200 len: 344 microversion: 2.1 time: 0.051858 May 19 12:40:39.303982 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1556/2992] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:39 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 344 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:39.546492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:39.547406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:39.554696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:39.555196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:39.556012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:39.556512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:39.574361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:39.694039 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:39.701756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-88c673c1-db39-4da6-a938-ce403fe27d02 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1508 microversion: 2.1 time: 0.159487 May 19 12:40:39.702292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1437/2993] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:39 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1508 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:40.194044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:40.195090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:40.202470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:40.202914 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:40.203664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:40.204093 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:40.267601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:40.275275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1a95de13-816d-4d00-89f8-c8ad07ccf835 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1241 microversion: 2.1 time: 0.088764 May 19 12:40:40.275849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1557/2994] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:40 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1241 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:40.315974 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-818ddbdc-5b9f-4d32-805f-6ab9bad8fffe tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c1919d0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:40.369562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-818ddbdc-5b9f-4d32-805f-6ab9bad8fffe tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 200 len: 344 microversion: 2.1 time: 0.057104 May 19 12:40:40.370149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1438/2995] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:40 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 344 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:40.723101 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:40.723862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:40.734389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:40.735097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:40.736282 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:40.737041 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:40.878378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:40.886761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-eaec3920-1679-40ef-a249-bec64f5f9eed tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1508 microversion: 2.1 time: 0.170453 May 19 12:40:40.887358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1558/2996] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:40 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1508 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:41.291353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.292300 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:41.299911 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:41.300360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:41.302118 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:41.302547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:41.370336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:41.382619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-fb80b680-76bd-41fb-ad06-7fae15e92677 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff4c4c0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.389232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cd63d13c-7de8-420f-8387-676fca2f5d4c tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1295 microversion: 2.1 time: 0.099004 May 19 12:40:41.389845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1439/2997] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1295 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:41.457854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-fb80b680-76bd-41fb-ad06-7fae15e92677 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 200 len: 345 microversion: 2.1 time: 0.079225 May 19 12:40:41.458890 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1559/2998] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 345 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:40:41.470105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-58141e7c-d2d9-4a8b-a206-dac94400b5b4 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Action: 'create', calling method: .version_select at 0x7fe20c1ca3a0>, body: {"snapshot": {"volume_id": "e1c9748e-9a4f-4916-b825-7c881469d131", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1079395108"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:41.588865 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-9e648eb0-d09c-4719-826d-9a3bff940e18 req-93b101c9-7570-4776-b026-4ebf1f088fcd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a9117bef-13c8-47cc-bb42-ea7a47d5025a", "name": "network-vif-deleted", "tag": "fc1f75e7-0085-4916-9009-12d6f71197eb"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:41.601229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9e648eb0-d09c-4719-826d-9a3bff940e18 req-93b101c9-7570-4776-b026-4ebf1f088fcd service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:41.602053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-9e648eb0-d09c-4719-826d-9a3bff940e18 req-93b101c9-7570-4776-b026-4ebf1f088fcd service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:41.618583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-9e648eb0-d09c-4719-826d-9a3bff940e18 req-93b101c9-7570-4776-b026-4ebf1f088fcd service nova] Creating event network-vif-deleted:fc1f75e7-0085-4916-9009-12d6f71197eb for instance a9117bef-13c8-47cc-bb42-ea7a47d5025a on hv-cinder-80403 May 19 12:40:41.624724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-9e648eb0-d09c-4719-826d-9a3bff940e18 req-93b101c9-7570-4776-b026-4ebf1f088fcd service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041596 May 19 12:40:41.626226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1560/2999] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:40:41.832462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-58141e7c-d2d9-4a8b-a206-dac94400b5b4 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.366277 May 19 12:40:41.837029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1440/3000] 10.222.0.35 () {60 vars in 1161 bytes} [Thu May 19 12:40:41 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:41.845042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d828cac1-fdd8-45fa-93b0-83d2e07bfd4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff4c160>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.881511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d828cac1-fdd8-45fa-93b0-83d2e07bfd4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.041123 May 19 12:40:41.882570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1561/3001] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:41.894697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f32338b4-c7af-4beb-a8f4-9310f5324cc4 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c407670>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.914541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.920907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:41.936620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:41.937120 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f32338b4-c7af-4beb-a8f4-9310f5324cc4 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.046719 May 19 12:40:41.937703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1441/3002] 10.222.0.35 () {58 vars in 1139 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:41.939796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:41.943456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:41.943929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:41.948097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-61872d4c-0eb6-45ae-92cb-dd0e929e9df2 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c131310>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:41.983409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-61872d4c-0eb6-45ae-92cb-dd0e929e9df2 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.039553 May 19 12:40:41.988137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1442/3003] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:40:42.013037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:42.013489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:42.013948 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'fault' on Instance uuid a9117bef-13c8-47cc-bb42-ea7a47d5025a {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:42.107896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:42.115572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-685815b0-eac4-446f-9c39-15064972f80f tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 200 len: 1311 microversion: 2.1 time: 0.216037 May 19 12:40:42.125995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1562/3004] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:41 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 1311 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:40:42.407043 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:42.407870 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:42.414486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:42.414929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:42.415741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:42.416366 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:42.551078 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:42.558996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2320e852-2596-464d-9458-d149540b74ea tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.156334 May 19 12:40:42.559570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1443/3005] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:42 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:42.887352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-dd651607-50bf-496c-96ac-40f45ac4df23 req-01e50a53-16b9-4cf4-83b2-832a149cd167 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cb97d22-79d2-4528-9565-90cc111227b2", "tag": "e59a0d70-625c-49cb-b7bc-09a37ab5e0cf"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:42.903287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-dd651607-50bf-496c-96ac-40f45ac4df23 req-01e50a53-16b9-4cf4-83b2-832a149cd167 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:42.903995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-dd651607-50bf-496c-96ac-40f45ac4df23 req-01e50a53-16b9-4cf4-83b2-832a149cd167 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:42.916530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-dd651607-50bf-496c-96ac-40f45ac4df23 req-01e50a53-16b9-4cf4-83b2-832a149cd167 service nova] Creating event network-changed:e59a0d70-625c-49cb-b7bc-09a37ab5e0cf for instance 2cb97d22-79d2-4528-9565-90cc111227b2 on hv-cinder-80403 May 19 12:40:42.921622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-dd651607-50bf-496c-96ac-40f45ac4df23 req-01e50a53-16b9-4cf4-83b2-832a149cd167 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041365 May 19 12:40:42.922465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1563/3006] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:42.995606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-38ac35e7-6dbb-4078-88e4-963343243baf tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c24f670>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:43.025404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-38ac35e7-6dbb-4078-88e4-963343243baf tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.033957 May 19 12:40:43.026153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1444/3007] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:42 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:43.133831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:43.134565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: a9117bef-13c8-47cc-bb42-ea7a47d5025a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:43.141421 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.141894 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.142646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.143070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.156580 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a HTTP/1.1 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:40:43.158246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. May 19 12:40:43.159991 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'a9117bef-13c8-47cc-bb42-ea7a47d5025a'} May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:43.160909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. May 19 12:40:43.162286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:43.163558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 404 to user: Instance a9117bef-13c8-47cc-bb42-ea7a47d5025a could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:43.163558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4e4199b3-69ee-4343-b652-5137ad6983cc tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a" status: 404 len: 111 microversion: 2.1 time: 0.030035 May 19 12:40:43.163558 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1564/3008] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:43 2022] GET /compute/v2.1/servers/a9117bef-13c8-47cc-bb42-ea7a47d5025a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:43.168306 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:43.168772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:43.175030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.175469 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.176223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.176689 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.209838 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:40:43.292617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2ef19c88-7dc2-44b0-802e-96b6cb3a7059 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "DELETE /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 204 len: 0 microversion: 2.1 time: 0.128039 May 19 12:40:43.293334 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1445/3009] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:40:43 2022] DELETE /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:40:43.301362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:43.301937 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:43.308065 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.308500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.309247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.309662 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.453786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:43.462051 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3e5afa16-cff9-4f76-8314-bcee5c5e188d tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.164509 May 19 12:40:43.462629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1565/3010] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:43 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:43.576606 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:43.577342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:43.584243 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.584676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.585462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:43.585890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:43.739318 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:43.747445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c0b12913-c749-47b1-b000-35646b3456dd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.175130 May 19 12:40:43.748010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1446/3011] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:43 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:44.037248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-58290ac2-44a7-4264-aca8-b8a4732302de tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fffa1f70>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:44.066866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-58290ac2-44a7-4264-aca8-b8a4732302de tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.033437 May 19 12:40:44.067429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1566/3012] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:44 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:44.483161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:44.484147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:44.494555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:44.494997 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:44.495748 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:44.496182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:44.630177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:44.638280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-020c9467-6488-4984-974a-465bec814806 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.161543 May 19 12:40:44.638835 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1447/3013] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:44 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:44.766216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:44.766856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:44.774231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:44.774745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:44.775625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:44.776125 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:44.912063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:44.921497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-529faa98-9b88-4b08-84f2-7f964b9561d4 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.159899 May 19 12:40:44.922035 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1567/3014] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:44 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:45.077928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5e154f5b-6972-4341-a5d9-94e71f3d66eb tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c3c61f0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:45.107406 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5e154f5b-6972-4341-a5d9-94e71f3d66eb tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.033275 May 19 12:40:45.107964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1448/3015] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:45 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:45.656364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:45.657081 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:45.665104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:45.665806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:45.666562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:45.666989 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:45.891462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:45.899768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8034f285-cd12-4d91-9f57-e9f89313bdc9 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1521 microversion: 2.1 time: 0.247906 May 19 12:40:45.900504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1568/3016] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:45 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1521 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:45.938908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:45.939523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:45.945809 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:45.946515 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:45.947445 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:45.947862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:46.119539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c14744c5-fbfd-43ac-87d4-31863d5c05de tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe211346ca0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:46.155283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c14744c5-fbfd-43ac-87d4-31863d5c05de tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 288 microversion: 2.1 time: 0.039830 May 19 12:40:46.156353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1569/3017] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:46 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 288 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:40:46.167564 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dacd2409-67eb-4f4d-a92d-1ef7014b57af tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff4c040>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:46.254514 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dacd2409-67eb-4f4d-a92d-1ef7014b57af tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "DELETE /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 202 len: 0 microversion: 2.1 time: 0.091594 May 19 12:40:46.255131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1570/3018] 10.222.0.35 () {58 vars in 1253 bytes} [Thu May 19 12:40:46 2022] DELETE /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 19 12:40:46.265648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-55f86692-cc5d-46ea-b058-97947632d201 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff6d9d0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:46.314140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-55f86692-cc5d-46ea-b058-97947632d201 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 200 len: 287 microversion: 2.1 time: 0.053970 May 19 12:40:46.314832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1571/3019] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:46 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 287 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:46.339620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-cf500b92-2cec-444f-bd4e-fae628592f71 req-cf9a4aca-e51b-46c2-b3b2-2341384c8fd7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da2ea452-9604-40b7-b9cb-65be5fa21be5", "name": "network-vif-deleted", "tag": "c5bf0325-0a37-4c7a-a88a-f713bf437054"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:46.351660 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-cf500b92-2cec-444f-bd4e-fae628592f71 req-cf9a4aca-e51b-46c2-b3b2-2341384c8fd7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:46.352189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-cf500b92-2cec-444f-bd4e-fae628592f71 req-cf9a4aca-e51b-46c2-b3b2-2341384c8fd7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:46.363876 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-cf500b92-2cec-444f-bd4e-fae628592f71 req-cf9a4aca-e51b-46c2-b3b2-2341384c8fd7 service nova] Creating event network-vif-deleted:c5bf0325-0a37-4c7a-a88a-f713bf437054 for instance da2ea452-9604-40b7-b9cb-65be5fa21be5 on hv-cinder-80403 May 19 12:40:46.370158 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-cf500b92-2cec-444f-bd4e-fae628592f71 req-cf9a4aca-e51b-46c2-b3b2-2341384c8fd7 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036213 May 19 12:40:46.371033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1572/3020] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:46.426625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:46.434730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a9f043fd-0aca-4a9e-8914-5bf356b3f133 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.500128 May 19 12:40:46.435297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1449/3021] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:45 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:46.919962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:46.920517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:46.927177 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:46.927617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:46.928357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:46.928817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:46.960142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:46.960576 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:46.961039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lazy-loading 'fault' on Instance uuid da2ea452-9604-40b7-b9cb-65be5fa21be5 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:40:47.002438 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '36415049be5d4238a0275db21a8f378a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce848ee51f841638902e08ea1c2af28', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:47.010504 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6a32327c-9e1a-49bb-bb8f-9b4c2575e883 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 200 len: 1311 microversion: 2.1 time: 0.096415 May 19 12:40:47.011337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1573/3022] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:46 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 1311 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:47.326088 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cdf5a57d-5dad-4919-b4b1-c018e700bb4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c1ca160>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:47.364187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-cdf5a57d-5dad-4919-b4b1-c018e700bb4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] HTTP exception thrown: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cdf5a57d-5dad-4919-b4b1-c018e700bb4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Request method failure captured: May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 HTTP/1.1 May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjrGNkcD-IuM8-MUVimSIF7qTeR778owSSEeSBOEgEnnWRJUCdbS7sFlQ9elSDegNw2nhcsB7GX_DadG0bd_PWAI9A7c-SeGxIAwMeU6UkMCTMztTupBGjN4ZqTRM1DYOTY_wmsQadyY9txW5kG44Q4B2_4owxsEQ--zvOdX9ADSEzo May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:47.367507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}] May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: a1887411e3ea4bd799b97f3536901212 May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'ae3a9022-ad60-4d8f-af60-44200af08128'} May 19 12:40:47.369887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. (HTTP 404) (Request-ID: req-85d15616-f0d9-4436-8a80-00e2fd6e46b6) May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.371028 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:47.372391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 19 12:40:47.373823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cdf5a57d-5dad-4919-b4b1-c018e700bb4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Returning 404 to user: Snapshot ae3a9022-ad60-4d8f-af60-44200af08128 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:47.375393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cdf5a57d-5dad-4919-b4b1-c018e700bb4a tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128" status: 404 len: 111 microversion: 2.1 time: 0.046907 May 19 12:40:47.376489 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1450/3023] 10.222.0.35 () {58 vars in 1250 bytes} [Thu May 19 12:40:47 2022] GET /compute/v2.1/os-snapshots/ae3a9022-ad60-4d8f-af60-44200af08128 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:47.379441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6cf0a065-d8b2-4ce3-b02f-55af37502f87 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff4cc10>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:47.445005 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6cf0a065-d8b2-4ce3-b02f-55af37502f87 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 202 len: 0 microversion: 2.1 time: 0.069307 May 19 12:40:47.445930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1574/3024] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:40:47 2022] DELETE /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:40:47.452660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:47.453652 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:47.463903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dc4c5fd7-5056-4708-bb65-58bfa2ee8603 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff6d9d0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:47.464067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:47.464067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:47.464067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:47.464067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:47.544068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dc4c5fd7-5056-4708-bb65-58bfa2ee8603 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 200 len: 344 microversion: 2.1 time: 0.092564 May 19 12:40:47.544068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1575/3025] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:47 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 344 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:47.624893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:47.632399 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2784f309-3665-4148-9b65-14582fe89cc0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.183897 May 19 12:40:47.632935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1451/3026] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:47 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:40:48.030402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:48.031037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] [instance: da2ea452-9604-40b7-b9cb-65be5fa21be5] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:48.039316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:48.039845 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:48.040769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:48.041274 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:48.057052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] HTTP exception thrown: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Request method failure captured: May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 HTTP/1.1 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKJAStPtgjfKjEPaIlWOr-Ebh67YOTpF8FWpto-U7kUucTFwP8hK9K-UD3zuA_SYllbGcwfqakeQI6FJwDdLf7ZZT-UVOHf0FV2S4E48cBLld1fk889wcOnGkgpfAFHfqOOqub4F7_vKRMnH2S4uBGU2-xDYu-pN2JoT7hWQFWEl8E May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:40:48.058682 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0ce848ee51f841638902e08ea1c2af28"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0ce848ee51f841638902e08ea1c2af28"}]}] May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0ce848ee51f841638902e08ea1c2af28 May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1835332371 May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 36415049be5d4238a0275db21a8f378a May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1835332371-project May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. May 19 12:40:48.060369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'da2ea452-9604-40b7-b9cb-65be5fa21be5'} May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:48.061374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. May 19 12:40:48.062996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.064456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] Returning 404 to user: Instance da2ea452-9604-40b7-b9cb-65be5fa21be5 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:48.064456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c8a2544e-2374-4d21-81d7-8641ebbf7a57 tempest-ServerRescueNegativeTestJSON-1835332371 tempest-ServerRescueNegativeTestJSON-1835332371-project] 10.222.0.35 "GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5" status: 404 len: 111 microversion: 2.1 time: 0.036354 May 19 12:40:48.064456 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1576/3027] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:48 2022] GET /compute/v2.1/servers/da2ea452-9604-40b7-b9cb-65be5fa21be5 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:48.556944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91b6a924-3c3c-4b38-ae71-b606541ff079 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c175a60>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:48.566207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:48.567242 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=10 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:48.588855 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-91b6a924-3c3c-4b38-ae71-b606541ff079 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] HTTP exception thrown: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91b6a924-3c3c-4b38-ae71-b606541ff079 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Request method failure captured: May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 HTTP/1.1 May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjrGNkcD-IuM8-MUVimSIF7qTeR778owSSEeSBOEgEnnWRJUCdbS7sFlQ9elSDegNw2nhcsB7GX_DadG0bd_PWAI9A7c-SeGxIAwMeU6UkMCTMztTupBGjN4ZqTRM1DYOTY_wmsQadyY9txW5kG44Q4B2_4owxsEQ--zvOdX9ADSEzo May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:48.590852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}] May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: a1887411e3ea4bd799b97f3536901212 May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'e1c9748e-9a4f-4916-b825-7c881469d131'} May 19 12:40:48.592722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. (HTTP 404) (Request-ID: req-0154228d-456f-4f49-a7a1-e488c516289d) May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.593922 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:48.595696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 19 12:40:48.597299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-91b6a924-3c3c-4b38-ae71-b606541ff079 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Returning 404 to user: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:48.598948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-91b6a924-3c3c-4b38-ae71-b606541ff079 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 404 len: 109 microversion: 2.1 time: 0.040009 May 19 12:40:48.600071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1452/3028] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:48 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 109 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:48.605222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4785afc-9a89-485c-ae84-3018d773da83 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe1fff6d700>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:48.611537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:48.635187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-a4785afc-9a89-485c-ae84-3018d773da83 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] HTTP exception thrown: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4785afc-9a89-485c-ae84-3018d773da83 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Request method failure captured: May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 HTTP/1.1 May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjrGNkcD-IuM8-MUVimSIF7qTeR778owSSEeSBOEgEnnWRJUCdbS7sFlQ9elSDegNw2nhcsB7GX_DadG0bd_PWAI9A7c-SeGxIAwMeU6UkMCTMztTupBGjN4ZqTRM1DYOTY_wmsQadyY9txW5kG44Q4B2_4owxsEQ--zvOdX9ADSEzo May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:40:48.637130 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}] May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: a1887411e3ea4bd799b97f3536901212 May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'e1c9748e-9a4f-4916-b825-7c881469d131'} May 19 12:40:48.638898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. (HTTP 404) (Request-ID: req-113053ad-33d6-4034-875d-fe192277b82a) May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.639963 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:40:48.641560 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 19 12:40:48.644753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4785afc-9a89-485c-ae84-3018d773da83 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Returning 404 to user: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:48.646306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a4785afc-9a89-485c-ae84-3018d773da83 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 404 len: 109 microversion: 2.1 time: 0.037368 May 19 12:40:48.647377 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1577/3029] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:40:48 2022] DELETE /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:40:48.648072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-83615c60-55ea-4d30-b133-b45ede6cb268 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Calling method '.version_select at 0x7fe20c1cae50>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:48.653783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:48.655266 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:48.661635 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:48.662572 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:48.663178 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:48.663620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:48.677604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-83615c60-55ea-4d30-b133-b45ede6cb268 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] HTTP exception thrown: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-83615c60-55ea-4d30-b133-b45ede6cb268 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Request method failure captured: May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 HTTP/1.1 May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjrGNkcD-IuM8-MUVimSIF7qTeR778owSSEeSBOEgEnnWRJUCdbS7sFlQ9elSDegNw2nhcsB7GX_DadG0bd_PWAI9A7c-SeGxIAwMeU6UkMCTMztTupBGjN4ZqTRM1DYOTY_wmsQadyY9txW5kG44Q4B2_4owxsEQ--zvOdX9ADSEzo May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:40:48.679545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_b61b511258b54d4ea26ca48bc5a2102f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/b61b511258b54d4ea26ca48bc5a2102f"}]}] May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: b61b511258b54d4ea26ca48bc5a2102f May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1433734577 May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: a1887411e3ea4bd799b97f3536901212 May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1433734577-project May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'e1c9748e-9a4f-4916-b825-7c881469d131'} May 19 12:40:48.681223 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. (HTTP 404) (Request-ID: req-b6c6c91b-712c-4888-9018-7b32bf2d6648) May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.685524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:40:48.687510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 19 12:40:48.689506 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-83615c60-55ea-4d30-b133-b45ede6cb268 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] Returning 404 to user: Volume e1c9748e-9a4f-4916-b825-7c881469d131 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-83615c60-55ea-4d30-b133-b45ede6cb268 tempest-VolumesSnapshotsTestJSON-1433734577 tempest-VolumesSnapshotsTestJSON-1433734577-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131" status: 404 len: 109 microversion: 2.1 time: 0.036688 May 19 12:40:48.692200 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1453/3030] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:48 2022] GET /compute/v2.1/os-volumes/e1c9748e-9a4f-4916-b825-7c881469d131 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 19 12:40:48.823921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:48.833896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5a3069d8-b4f5-484e-995c-e624b500c1b5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.184027 May 19 12:40:48.834649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1578/3031] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:48 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:49.852008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:49.852604 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:49.865428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:49.866305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:49.867361 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:49.868227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:50.059834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:50.075882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fff1c536-45b4-43a1-8c7a-9238063fb03e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.224157 May 19 12:40:50.082184 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1454/3032] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:49 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:40:51.089533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:51.090699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:51.097952 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:51.098562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:51.099327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:51.099750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:51.514337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:51.523310 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5e22dc79-bf0c-4d92-b54c-0207c1d890ee tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.437889 May 19 12:40:51.524096 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1579/3033] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:51 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:51.636486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-df6fba97-f045-44f3-baf0-af7e1d07b03a req-0c698ce5-ca93-47f2-9e09-b8ce291892db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cb97d22-79d2-4528-9565-90cc111227b2", "name": "network-vif-plugged", "status": "completed", "tag": "e59a0d70-625c-49cb-b7bc-09a37ab5e0cf"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:51.646819 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-df6fba97-f045-44f3-baf0-af7e1d07b03a req-0c698ce5-ca93-47f2-9e09-b8ce291892db service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:51.647548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-df6fba97-f045-44f3-baf0-af7e1d07b03a req-0c698ce5-ca93-47f2-9e09-b8ce291892db service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:51.663047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-df6fba97-f045-44f3-baf0-af7e1d07b03a req-0c698ce5-ca93-47f2-9e09-b8ce291892db service nova] Creating event network-vif-plugged:e59a0d70-625c-49cb-b7bc-09a37ab5e0cf for instance 2cb97d22-79d2-4528-9565-90cc111227b2 on hv-cinder-80403 May 19 12:40:51.669163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-df6fba97-f045-44f3-baf0-af7e1d07b03a req-0c698ce5-ca93-47f2-9e09-b8ce291892db service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036463 May 19 12:40:51.669872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1455/3034] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:51.719508 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-227c1768-f90b-4d62-ae9c-e60d1605cbd4 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe21134d040>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:51.782308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-227c1768-f90b-4d62-ae9c-e60d1605cbd4 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/bfb16477-d0d7-4a55-bafd-291629518670" status: 200 len: 351 microversion: 2.1 time: 0.067020 May 19 12:40:51.789085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1580/3035] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:51 2022] GET /compute/v2.1/os-volumes/bfb16477-d0d7-4a55-bafd-291629518670 => generated 351 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:52.541933 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:52.542620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:52.549721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:52.550183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:52.551127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:52.551555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:52.687414 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:52.695066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-61ebb58b-5a38-4974-a9d0-c1a209805ac6 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1363 microversion: 2.1 time: 0.157314 May 19 12:40:52.695617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1456/3036] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:52 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:53.213803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-86c06fd4-199b-46c4-960e-d8f06e55f29c tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe1fff76160>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:53.269731 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-86c06fd4-199b-46c4-960e-d8f06e55f29c tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/fa0885e7-aa81-4308-95e9-aeed867da682" status: 200 len: 351 microversion: 2.1 time: 0.060680 May 19 12:40:53.270811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1581/3037] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:53 2022] GET /compute/v2.1/os-volumes/fa0885e7-aa81-4308-95e9-aeed867da682 => generated 351 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:53.711329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:53.711934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:53.723593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:53.724040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:53.724843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:53.725276 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:54.025698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:40:54.034148 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-85f0bd2f-e11f-4ca5-ae24-127c11958394 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1548 microversion: 2.1 time: 0.326595 May 19 12:40:54.034715 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1457/3038] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:40:53 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1548 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:54.050920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:54.051756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:40:54.060258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:54.060720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:54.061485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:54.061906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:54.148236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-010fc8d4-30ea-4253-befa-bdd7ceb6011f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.100913 May 19 12:40:54.148958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1582/3039] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:40:54 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-interface => generated 284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:54.669173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bf25adda-8d15-46f5-95e9-3cb8796d9857 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe20c1ca700>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:54.728283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bf25adda-8d15-46f5-95e9-3cb8796d9857 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/d43bf81d-11d1-4f51-b78a-24813231a060" status: 200 len: 351 microversion: 2.1 time: 0.062961 May 19 12:40:54.729035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1458/3040] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:40:54 2022] GET /compute/v2.1/os-volumes/d43bf81d-11d1-4f51-b78a-24813231a060 => generated 351 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:54.741465 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aa69883d-a95c-4d71-b0fc-c2e750d87b42 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe1fff6dd30>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:54.831079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aa69883d-a95c-4d71-b0fc-c2e750d87b42 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.094247 May 19 12:40:54.832190 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1583/3041] 10.222.0.35 () {58 vars in 1133 bytes} [Thu May 19 12:40:54 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:54.844249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5f238e8c-1798-4364-8a07-be42bc49e81f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe20c586f70>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:54.926560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5f238e8c-1798-4364-8a07-be42bc49e81f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.086081 May 19 12:40:54.927135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1459/3042] 10.222.0.35 () {58 vars in 1148 bytes} [Thu May 19 12:40:54 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:54.940223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d6b3f0e1-17f6-4b8e-aa4f-5e9dd2e6ea0b tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe21134d940>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.025267 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d6b3f0e1-17f6-4b8e-aa4f-5e9dd2e6ea0b tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.088836 May 19 12:40:55.026231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1584/3043] 10.222.0.35 () {58 vars in 1133 bytes} [Thu May 19 12:40:54 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:55.037743 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-26016f09-47e6-4f28-af8a-83fd885db04d tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe20c407430>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.120523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-26016f09-47e6-4f28-af8a-83fd885db04d tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.086414 May 19 12:40:55.121139 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1460/3044] 10.222.0.35 () {58 vars in 1166 bytes} [Thu May 19 12:40:55 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:55.133753 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a4d3f117-c941-4b7f-a40f-b044268cf80f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe21136a430>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.259046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a4d3f117-c941-4b7f-a40f-b044268cf80f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.128970 May 19 12:40:55.260147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1585/3045] 10.222.0.35 () {58 vars in 1169 bytes} [Thu May 19 12:40:55 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:55.272716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-580d6af4-8bbf-4ea9-9327-b40d8de76bab tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe20c4078b0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.346711 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-17ad1376-b6e5-4096-ae5b-da506fd73abc req-474a2534-3199-4c93-8132-15967b4868b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cb97d22-79d2-4528-9565-90cc111227b2", "tag": "e59a0d70-625c-49cb-b7bc-09a37ab5e0cf"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:55.365909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-17ad1376-b6e5-4096-ae5b-da506fd73abc req-474a2534-3199-4c93-8132-15967b4868b5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:40:55.366899 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-17ad1376-b6e5-4096-ae5b-da506fd73abc req-474a2534-3199-4c93-8132-15967b4868b5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:40:55.404502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-17ad1376-b6e5-4096-ae5b-da506fd73abc req-474a2534-3199-4c93-8132-15967b4868b5 service nova] Creating event network-changed:e59a0d70-625c-49cb-b7bc-09a37ab5e0cf for instance 2cb97d22-79d2-4528-9565-90cc111227b2 on hv-cinder-80403 May 19 12:40:55.422657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-17ad1376-b6e5-4096-ae5b-da506fd73abc req-474a2534-3199-4c93-8132-15967b4868b5 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076354 May 19 12:40:55.425609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1586/3046] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:40:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:55.457948 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-580d6af4-8bbf-4ea9-9327-b40d8de76bab tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.188998 May 19 12:40:55.458517 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1461/3047] 10.222.0.35 () {58 vars in 1154 bytes} [Thu May 19 12:40:55 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:40:55.470707 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-73f837c1-437e-4f2f-a2ab-907b1b75113f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe21136a9d0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.604860 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-73f837c1-437e-4f2f-a2ab-907b1b75113f tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.138183 May 19 12:40:55.605636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1587/3048] 10.222.0.35 () {58 vars in 1187 bytes} [Thu May 19 12:40:55 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:40:55.618007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4e31a3cf-eac4-4512-bf97-4ef6244665f5 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] Calling method '.version_select at 0x7fe1fff971f0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:40:55.733319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4e31a3cf-eac4-4512-bf97-4ef6244665f5 tempest-VolumesTestJSON-1332385338 tempest-VolumesTestJSON-1332385338-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.120049 May 19 12:40:55.734849 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1462/3049] 10.222.0.35 () {58 vars in 1154 bytes} [Thu May 19 12:40:55 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:40:58.659287 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0ffc0365-ec03-4ae7-b8bd-2205727e4aa5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: .version_select at 0x7fe20cbc8dc0>, body: {"keypair": {"name": "tempest-keypair-1850550406", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCgMzczNq6lubqRyaoCTjnA0/GdQ3Ek2OCJjzIhRp4E9dGSBeb84EXNVONPs24peOau2BvAOAbQQPblLZSRTpu769FqsPXS8L7ZsJXm4G5zJJomqbH3/bl/XqvEjImE9eA=="}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:40:58.683561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-0ffc0365-ec03-4ae7-b8bd-2205727e4aa5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:40:58.687802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-0ffc0365-ec03-4ae7-b8bd-2205727e4aa5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:40:58.710988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0ffc0365-ec03-4ae7-b8bd-2205727e4aa5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.208827 May 19 12:40:58.711595 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1588/3050] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:40:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:41:00.443853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1771934989", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f504b23c-8d0f-4020-83c4-474e12095c43"}], "security_groups": [{"name": "tempest-securitygroup--63003558"}], "key_name": "tempest-keypair-1850550406", "user_data": "}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:00.681101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] validate_networks() for [('f504b23c-8d0f-4020-83c4-474e12095c43', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:41:01.315559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:41:01.315987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:41:01.316460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:41:01.316941 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:41:01.317280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:41:01.334167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:01.334642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:01.335433 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:01.335866 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:01.376498 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:41:01.381159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:41:01.403345 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:41:01.406319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:41:01.487553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-63f4ee9a-bc5e-4ee8-a8ab-49a6e9f6bf1c tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 1.048817 May 19 12:41:01.488432 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1463/3051] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:41:00 2022] POST /compute/v2.1/servers => generated 400 bytes in 1052 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:41:01.510141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-99483010-b3bb-419b-b692-d0767fc8650f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:01.510141 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-99483010-b3bb-419b-b692-d0767fc8650f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:01.625906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-99483010-b3bb-419b-b692-d0767fc8650f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:01.634324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-99483010-b3bb-419b-b692-d0767fc8650f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1242 microversion: 2.1 time: 0.139335 May 19 12:41:01.635234 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1589/3052] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:01 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1242 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:02.657584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:02.658540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:02.678151 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:02.680586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:02.681451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:02.682083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:02.801285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:02.810317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15de2efd-3fe9-4a7e-82d1-7c8381cbc47a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1294 microversion: 2.1 time: 0.162298 May 19 12:41:02.811451 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1464/3053] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:02 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1294 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:03.827620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:03.828233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:03.838960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:03.839144 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:03.839788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:03.840214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:03.952979 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:03.961548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c7b1e8b8-0c60-46c6-8c72-fca1e54322f4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1300 microversion: 2.1 time: 0.137926 May 19 12:41:03.962955 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1590/3054] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:03 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1300 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:04.979352 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:04.980624 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:04.983418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=7,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:04.996356 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:04.997297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:04.998393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:04.998985 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:05.276512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:05.306729 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cd229d27-d390-49e6-93dc-13a40d8dfaa8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.331205 May 19 12:41:05.318539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1465/3055] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:04 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:05.403426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-68a96694-7c8d-45f3-8837-216dbf32e4da req-566ad187-7471-4343-8c44-2529c959ee87 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4440209e-114a-4e00-866c-006bba2d62fa", "tag": "17d404c2-5bde-4e15-8153-61e1db648429"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:05.427698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:05.448262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-68a96694-7c8d-45f3-8837-216dbf32e4da req-566ad187-7471-4343-8c44-2529c959ee87 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:05.449042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-68a96694-7c8d-45f3-8837-216dbf32e4da req-566ad187-7471-4343-8c44-2529c959ee87 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:05.463522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-68a96694-7c8d-45f3-8837-216dbf32e4da req-566ad187-7471-4343-8c44-2529c959ee87 service nova] Creating event network-changed:17d404c2-5bde-4e15-8153-61e1db648429 for instance 4440209e-114a-4e00-866c-006bba2d62fa on hv-cinder-80403 May 19 12:41:05.469419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-68a96694-7c8d-45f3-8837-216dbf32e4da req-566ad187-7471-4343-8c44-2529c959ee87 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072142 May 19 12:41:05.474469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1591/3056] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:06.327094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:06.327713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:06.335579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:06.336303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:06.337601 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:06.338188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:06.524990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:06.533775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a5e96f30-273a-4248-a9da-09be5c4d8d09 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.211062 May 19 12:41:06.534351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1466/3057] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:06 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:07.555762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:07.556791 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:07.565795 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:07.566245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:07.567405 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:07.567832 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:07.574378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:07.815541 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:07.826318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-94f54fe3-1ffe-48cf-8cb6-6b29c61fdb1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.277811 May 19 12:41:07.827149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1592/3058] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:07 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:08.843430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:08.844112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:08.857295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:08.857740 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:08.858533 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:08.864501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:09.057611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:09.059611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:09.064330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5d32a1cd-aff6-4d6c-8562-4c24bc1a8992 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.224792 May 19 12:41:09.065584 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1467/3059] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:08 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:10.083303 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:10.083907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:10.094124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:10.094559 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:10.095320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:10.095740 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:10.365797 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:10.375714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dc08c374-9493-459d-817c-7ec0f4f1923f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.296324 May 19 12:41:10.376304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1593/3060] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:10 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:11.392680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:11.393924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:11.396990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:11.398128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:11.405075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:11.405916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:11.406960 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:11.407586 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:11.662075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-b034369f-e776-43fa-a664-1bb15404510c req-667c3624-f7e6-4379-8266-25828410df95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4440209e-114a-4e00-866c-006bba2d62fa", "name": "network-vif-plugged", "status": "completed", "tag": "17d404c2-5bde-4e15-8153-61e1db648429"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:11.673413 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:11.675818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b034369f-e776-43fa-a664-1bb15404510c req-667c3624-f7e6-4379-8266-25828410df95 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:11.676376 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b034369f-e776-43fa-a664-1bb15404510c req-667c3624-f7e6-4379-8266-25828410df95 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:11.695246 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-b034369f-e776-43fa-a664-1bb15404510c req-667c3624-f7e6-4379-8266-25828410df95 service nova] Creating event network-vif-plugged:17d404c2-5bde-4e15-8153-61e1db648429 for instance 4440209e-114a-4e00-866c-006bba2d62fa on hv-cinder-80403 May 19 12:41:11.705625 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-b034369f-e776-43fa-a664-1bb15404510c req-667c3624-f7e6-4379-8266-25828410df95 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046979 May 19 12:41:11.706614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1594/3061] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:12.514164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:12.532272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b0b800ba-70df-46d4-985b-1365641ff635 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 1.143650 May 19 12:41:12.548949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1468/3062] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:11 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 1161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:13.557176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:13.557904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:13.566756 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:13.567136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:13.568269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:13.568573 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:13.802754 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:13.814364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-63a52464-336e-40ab-b036-1af3885b5e56 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1366 microversion: 2.1 time: 0.260636 May 19 12:41:13.816806 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1595/3063] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:13 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1366 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:14.831814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:14.832504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:14.839734 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:14.840398 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:14.841193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:14.841850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:15.049683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:15.060615 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28a6d8b2-31fb-4145-a5a0-6f24e51f5ad0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1556 microversion: 2.1 time: 0.232698 May 19 12:41:15.062811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1469/3064] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:14 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1556 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:15.087854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:15.089322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:15.097036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:15.097714 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:15.098670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:15.099472 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:15.205358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e37665a5-df08-4c75-843d-3f6f8957480a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.127550 May 19 12:41:15.213292 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1596/3065] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:41:15 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-interface => generated 284 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:16.619434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-0c3e284f-8c04-48db-9101-41c3af9c038e req-35555ae4-738d-4dcc-9384-608b3c87ce15 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4440209e-114a-4e00-866c-006bba2d62fa", "tag": "17d404c2-5bde-4e15-8153-61e1db648429"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:16.642391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0c3e284f-8c04-48db-9101-41c3af9c038e req-35555ae4-738d-4dcc-9384-608b3c87ce15 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:16.643619 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-0c3e284f-8c04-48db-9101-41c3af9c038e req-35555ae4-738d-4dcc-9384-608b3c87ce15 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:16.662982 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-0c3e284f-8c04-48db-9101-41c3af9c038e req-35555ae4-738d-4dcc-9384-608b3c87ce15 service nova] Creating event network-changed:17d404c2-5bde-4e15-8153-61e1db648429 for instance 4440209e-114a-4e00-866c-006bba2d62fa on hv-cinder-80403 May 19 12:41:16.669341 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-0c3e284f-8c04-48db-9101-41c3af9c038e req-35555ae4-738d-4dcc-9384-608b3c87ce15 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054058 May 19 12:41:16.671103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1470/3066] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:23.467669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-775368db-c46c-4984-9d26-564d11091bd1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'create', calling method: .version_select at 0x7fe21131eca0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1665217945", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIDRo528GaFpwjnKgnSDXXTjqD39YxISaKG9nsI5kROwqsn8iRvfky7mJuXEQff+GXoJzESnOWSWPa3KQriy+IycX3jb3Beui5/Y3Nh614Qxia1tEX1xfqRL/yLv55FysA=="}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:23.482309 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-775368db-c46c-4984-9d26-564d11091bd1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Getting quotas for project a76f72af8d47404980293788d97edb2e. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:41:23.486249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-775368db-c46c-4984-9d26-564d11091bd1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Getting quotas for user f725498599d84b3782c0e0d46131463a and project a76f72af8d47404980293788d97edb2e. Resources: {'key_pairs'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:41:23.518063 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-775368db-c46c-4984-9d26-564d11091bd1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.054821 May 19 12:41:23.518778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1597/3067] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:41:23 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:23.628996 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1758108938", "imageRef": "0745d0d8-3720-427f-9775-8c2548d9f932", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1665217945", "networks": [{"uuid": "b4364b4e-2076-4b42-ae24-05ad70b4107c"}]}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:23.682232 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] validate_networks() for [('b4364b4e-2076-4b42-ae24-05ad70b4107c', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:41:24.060394 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:41:24.060910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:41:24.061465 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:41:24.061930 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:41:24.062396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:41:24.071550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:24.072054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:24.072926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:24.073568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:24.090947 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Getting quotas for project a76f72af8d47404980293788d97edb2e. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:41:24.094884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Getting quotas for user f725498599d84b3782c0e0d46131463a and project a76f72af8d47404980293788d97edb2e. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:41:24.106579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:41:24.108888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='0745d0d8-3720-427f-9775-8c2548d9f932',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:41:24.154753 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7773b6da-9d80-4f0b-852f-4287b91f7015 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.530004 May 19 12:41:24.160052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1471/3068] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:41:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 532 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:41:24.166161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4a1fe2b1-0eae-4a7b-81bb-18b62f8a1b84 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:24.167232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4a1fe2b1-0eae-4a7b-81bb-18b62f8a1b84 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:24.263290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4a1fe2b1-0eae-4a7b-81bb-18b62f8a1b84 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:24.479460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4a1fe2b1-0eae-4a7b-81bb-18b62f8a1b84 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1259 microversion: 2.1 time: 0.317895 May 19 12:41:24.481025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1598/3069] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:24 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1259 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:25.498871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:25.499839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:25.508177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:25.508798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:25.509800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:25.510526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:25.593372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:25.601105 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab1c4783-9504-4167-b7e6-4d1f8d286f93 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1317 microversion: 2.1 time: 0.107365 May 19 12:41:25.601845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1472/3070] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:25 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1317 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:26.619218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:26.620627 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:26.630827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=23 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:26.642189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:26.642916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:26.643871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:26.645314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:26.875221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:26.903480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-445fdddf-c787-42e2-938f-ee75edcdbd7d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.288487 May 19 12:41:26.906078 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1599/3071] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:26 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:27.921752 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:27.922337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:27.928935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:27.929374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:27.930125 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:27.930595 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:28.066524 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:28.077654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e3b6873c-95ab-455d-aea2-27a87504749e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.159546 May 19 12:41:28.078407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1473/3072] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:27 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:29.099402 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:29.100195 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:29.107069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:29.107519 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:29.108275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:29.108727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:29.256561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:29.267757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-38f0c236-a795-4c88-b8a5-e38b896ad84d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.172636 May 19 12:41:29.268966 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1600/3073] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:29 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:30.285153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:30.285841 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:30.293054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:30.293769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:30.294536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:30.295101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:30.460263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:30.472968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1a296587-7e8d-496c-adbd-515ca81cf9be tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.191720 May 19 12:41:30.473555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1474/3074] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:30 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:31.490346 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:31.490992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:31.500877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:31.500877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:31.500877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:31.500877 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:31.676586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:31.684258 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cfe08706-721b-4571-9a53-96749051c688 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.197761 May 19 12:41:31.684857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1601/3075] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:31 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:32.581309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:32.581973 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:32.589343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:32.589793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:32.590704 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:32.591179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:32.614887 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ef1b9de1-ef48-4ea3-8c04-9909483314df tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/ips" status: 200 len: 144 microversion: 2.1 time: 0.037534 May 19 12:41:32.615454 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1475/3076] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:41:32 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/ips => generated 144 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:32.702294 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:32.703932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:32.712838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:32.714164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:32.715360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:32.716109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:32.884933 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:32.897052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bf3aa009-11ec-4f7d-81f5-2b272535fac2 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.199386 May 19 12:41:32.897992 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1602/3077] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:32 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:33.917698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:33.919036 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:33.927359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:33.928124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:33.929588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:33.930585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:34.074603 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:34.081187 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:34.084787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-987909aa-d839-4f09-b8ac-84ce1a79dddf tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.173320 May 19 12:41:34.095090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1476/3078] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:33 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:34.129720 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "690cf582-d885-4ba5-a23c-ade98c6a6f65"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:34.130771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:34.142326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:34.142766 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:34.143532 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:34.143980 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:34.269412 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:35.099840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:35.105612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:35.106666 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:35.108756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:35.115224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:35.115940 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:35.117164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:35.117952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:35.330482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:35.338455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b2771680-9109-4164-8b74-5c1d9385ca6d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.239937 May 19 12:41:35.339149 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1477/3079] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:35 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:36.356957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:36.357755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:36.366023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:36.366463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:36.367215 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:36.367638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:36.511064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:36.518589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e8f34516-532b-4433-9ee0-1084597c6a5d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.167159 May 19 12:41:36.519137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1478/3080] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:36 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:41:37.538216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:37.538816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:37.548430 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:37.548972 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:37.549735 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:37.550164 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:37.679637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:37.687631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fc4d6f51-8eb0-4e50-892a-5a5ad4ecae67 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.155710 May 19 12:41:37.688198 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1479/3081] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:37 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:38.710058 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:38.711342 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:38.723660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:38.724578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:38.725703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:38.726421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:38.799471 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-78000731-a178-4b39-9857-0c1b77644596 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.673558 May 19 12:41:38.800181 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1603/3082] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:41:34 2022] POST /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 194 bytes in 4675 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:38.969808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:38.977883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bac1ae91-d58c-4853-b336-b09cbe3f683b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.276673 May 19 12:41:38.978472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1480/3083] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:38 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:41:39.325214 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-be6dd136-0518-4347-80f7-8b47f448ea56 req-86e91604-ac0b-4a66-ba86-07a940e7d59e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:39.334019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:39.338589 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-be6dd136-0518-4347-80f7-8b47f448ea56 req-86e91604-ac0b-4a66-ba86-07a940e7d59e service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:39.339270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-be6dd136-0518-4347-80f7-8b47f448ea56 req-86e91604-ac0b-4a66-ba86-07a940e7d59e service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:39.358355 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-be6dd136-0518-4347-80f7-8b47f448ea56 req-86e91604-ac0b-4a66-ba86-07a940e7d59e service nova] Creating event network-changed:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:41:39.363061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-be6dd136-0518-4347-80f7-8b47f448ea56 req-86e91604-ac0b-4a66-ba86-07a940e7d59e service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042319 May 19 12:41:39.364245 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1604/3084] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:39.996179 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:39.996762 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:40.003573 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:40.004013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:40.004791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:40.005216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:40.146077 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:40.157495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-edcce2ea-3ba1-4142-a17a-3e987534cd02 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.165494 May 19 12:41:40.158347 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1481/3085] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:39 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:41.182349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:41.184042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:41.194182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:41.194919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:41.196042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:41.196588 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:41.331680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:41.339209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8b010cd9-e71c-4085-b26a-105b2f40528f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.161992 May 19 12:41:41.339767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1605/3086] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:41 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:42.356380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:42.357161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:42.364013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:42.364447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:42.365237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:42.365660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:42.368456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=75 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:42.529395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:42.537003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ef0bbc45-bbab-4783-bb34-cd9d7c72c8cb tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.185102 May 19 12:41:42.537633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1482/3087] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:42 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:43.555039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:43.556423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:43.564545 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:43.565149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:43.565945 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:43.566369 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:43.586091 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-2a616aae-4088-486b-adf8-605de3b2d1da req-b7d84594-0b07-496f-8f8f-e7a9bddd5ec9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "name": "network-vif-plugged", "status": "completed", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:43.595692 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-2a616aae-4088-486b-adf8-605de3b2d1da req-b7d84594-0b07-496f-8f8f-e7a9bddd5ec9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:43.596191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-2a616aae-4088-486b-adf8-605de3b2d1da req-b7d84594-0b07-496f-8f8f-e7a9bddd5ec9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:43.611501 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-2a616aae-4088-486b-adf8-605de3b2d1da req-b7d84594-0b07-496f-8f8f-e7a9bddd5ec9 service nova] Creating event network-vif-plugged:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:41:43.619500 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-2a616aae-4088-486b-adf8-605de3b2d1da req-b7d84594-0b07-496f-8f8f-e7a9bddd5ec9 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036818 May 19 12:41:43.620750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1483/3088] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:43.702586 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:43.710605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-163c851a-2a1a-4e50-8b25-727fdcb2a2a1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.161112 May 19 12:41:43.711249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1606/3089] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:43 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:44.728100 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:44.728837 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:44.735963 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:44.736391 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:44.737173 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:44.737719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:44.873097 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:44.880897 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bae8ca92-54c3-4e84-a6b4-687d1fdb3d4b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1359 microversion: 2.1 time: 0.157050 May 19 12:41:44.881480 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1484/3090] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:44 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1359 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:45.898216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:45.898936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:45.906304 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:45.906755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:45.907526 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:45.907959 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.048305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:46.057591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-374891b1-b18f-4f42-b692-e543dde75344 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1548 microversion: 2.1 time: 0.163393 May 19 12:41:46.058145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1607/3091] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:45 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1548 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:46.076535 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:46.077413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:46.087486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.087932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.088746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.089180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.234065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:46.242145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a56f8859-39e3-41aa-9924-174e80422d3e tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1548 microversion: 2.1 time: 0.171999 May 19 12:41:46.243284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1485/3092] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:46 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1548 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:46.259064 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:46.262664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:46.263458 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Searching by: {'deleted': False, 'project_id': 'a76f72af8d47404980293788d97edb2e'} {{(pid=117533) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 19 12:41:46.270975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.271416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.272197 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.272626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.363466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.multi_cell_list [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117533) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 19 12:41:46.459609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b4e5479f-3075-41fc-af55-53146eff192f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.204352 May 19 12:41:46.460352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1608/3093] 10.222.0.35 () {58 vars in 1124 bytes} [Thu May 19 12:41:46 2022] GET /compute/v2.1/servers => generated 343 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:46.477791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:46.478359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:46.484861 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.485304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.486046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:46.486470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:46.630568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:46.638955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-fa913e48-27f1-4be2-a601-d0ec15d9596d tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1548 microversion: 2.1 time: 0.168887 May 19 12:41:46.639546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1486/3094] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:46 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1548 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:47.407525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:47.409156 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:47.416084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:47.416533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:47.417324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:47.417767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:47.447974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a074699f-3ba8-440d-803e-ebc8a1449570 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044653 May 19 12:41:47.448539 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1609/3095] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:41:47 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:47.458956 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:47.459554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:47.466424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:47.466914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:47.467646 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:47.468075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:47.496621 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7952a068-959a-46ef-af99-11ef4809c917 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65" status: 200 len: 194 microversion: 2.1 time: 0.041731 May 19 12:41:47.497233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1487/3096] 10.222.0.35 () {58 vars in 1412 bytes} [Thu May 19 12:41:47 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:48.214389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5239592d-4a9e-4fe4-bbd3-859e6a01f961"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:48.215524 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:48.221669 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:48.222191 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:48.223030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:48.223534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:48.501517 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aaf25108-e295-4dac-994b-cc37b5102aac tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.292064 May 19 12:41:48.502450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1610/3097] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:41:48 2022] POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments => generated 194 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:48.846137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "dc745524-c0da-4062-b626-cd00d226360c"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:48.847525 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:48.857453 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:48.857842 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:48.859005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:48.859593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:49.139377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-929af43e-5964-4917-8eaa-af772db9a01e tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "POST /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.299476 May 19 12:41:49.140015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1488/3098] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:41:48 2022] POST /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 194 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:56.133161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:56.133951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:56.140182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:56.140622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:56.141407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:56.141834 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:56.278468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:56.286976 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4d0a3a70-c2ce-46cf-bee7-6845b37a446c tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1594 microversion: 2.1 time: 0.158672 May 19 12:41:56.287542 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1611/3099] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:56 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1594 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:41:58.078828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-e3cc1120-3150-4893-a4df-6ffb40e3f537 req-c041f76f-5f28-492f-abab-b753f3c1f496 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:58.097914 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e3cc1120-3150-4893-a4df-6ffb40e3f537 req-c041f76f-5f28-492f-abab-b753f3c1f496 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:58.099852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e3cc1120-3150-4893-a4df-6ffb40e3f537 req-c041f76f-5f28-492f-abab-b753f3c1f496 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:58.126245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-e3cc1120-3150-4893-a4df-6ffb40e3f537 req-c041f76f-5f28-492f-abab-b753f3c1f496 service nova] Creating event network-changed:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:41:58.132744 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-e3cc1120-3150-4893-a4df-6ffb40e3f537 req-c041f76f-5f28-492f-abab-b753f3c1f496 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065212 May 19 12:41:58.134289 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1489/3100] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:41:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:59.077655 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1185963528"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:41:59.078247 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:59.080273 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:59.086052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.086527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.087283 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.087706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.090581 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:59.149136 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:59.149714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:59.160789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.161244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.162015 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.163326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.186667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-07ff92c2-83dd-4cbb-ab70-2047c46f91b0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/ips" status: 200 len: 150 microversion: 2.1 time: 0.041709 May 19 12:41:59.187256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1490/3101] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:41:59 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:41:59.228808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.network.security_group_api [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Adding security group 1c5346d4-b915-4fd9-aeb2-38ba58129fd8 to port fedd35b6-9b61-414c-b8f9-173f39f64c6e May 19 12:41:59.625958 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-dcdca0be-2fc7-48df-b1d4-782eccdc30fa tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action" status: 202 len: 0 microversion: 2.1 time: 0.552203 May 19 12:41:59.626649 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1612/3102] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:41:59 2022] POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action => generated 0 bytes in 554 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:41:59.635228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:41:59.635843 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:41:59.642798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.643177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.643921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:41:59.644375 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:41:59.800412 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:41:59.808467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-42f31c05-8d6f-454b-8caf-fbadec7f11d0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1641 microversion: 2.1 time: 0.177432 May 19 12:41:59.809059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1491/3103] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:41:59 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1641 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:00.149404 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-95feb53b-32b8-439d-a5d4-06a88a86f64b req-1b9091dc-05f6-4790-b84b-2113bfc03459 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:00.157928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-95feb53b-32b8-439d-a5d4-06a88a86f64b req-1b9091dc-05f6-4790-b84b-2113bfc03459 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:00.158362 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-95feb53b-32b8-439d-a5d4-06a88a86f64b req-1b9091dc-05f6-4790-b84b-2113bfc03459 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:00.169757 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-95feb53b-32b8-439d-a5d4-06a88a86f64b req-1b9091dc-05f6-4790-b84b-2113bfc03459 service nova] Creating event network-changed:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:42:00.174944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-95feb53b-32b8-439d-a5d4-06a88a86f64b req-1b9091dc-05f6-4790-b84b-2113bfc03459 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029450 May 19 12:42:00.175544 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1613/3104] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:42:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:02.680482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "428a739e-a87d-4756-ac58-ad45c67b7f37"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:02.681792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:02.690523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:02.690970 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:02.691763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:02.692186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:12.228146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5127e904-ec93-4c70-84ff-127ffd33af85 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 9.553713 May 19 12:42:12.228928 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1492/3105] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:42:02 2022] POST /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments => generated 194 bytes in 9556 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:13.647736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=19 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:13.648493 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=5 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:13.653018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:13.654102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:13.661173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.661621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.662371 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.662808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.692916 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6a9375cc-f89c-4702-8da5-eda2dd9f54d7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.046596 May 19 12:42:13.693525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1614/3106] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:42:13 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 371 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:13.838377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=31 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:13.842682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:13.843261 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:13.850009 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.850458 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.851207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.851669 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.854059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:13.883590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1c2e46d6-bf66-4e84-a963-ee33aaf304f7 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65" status: 200 len: 194 microversion: 2.1 time: 0.046651 May 19 12:42:13.884171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1493/3107] 10.222.0.35 () {58 vars in 1412 bytes} [Thu May 19 12:42:13 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:13.897353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:13.898012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:13.907032 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.907617 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.908664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:13.909142 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:13.991203 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'services' on Instance uuid 2cb97d22-79d2-4528-9565-90cc111227b2 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:42:14.067313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b67b237a-e82f-4849-95d0-44c5d49ea8ae tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65" status: 202 len: 0 microversion: 2.1 time: 0.176299 May 19 12:42:14.067922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1615/3108] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:42:13 2022] DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/690cf582-d885-4ba5-a23c-ade98c6a6f65 => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:42:20.831117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6126ff58-0a8a-4d5b-98d7-faba413eb861 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: .version_select at 0x7fe1fff650d0>, body: {"keypair": {"name": "tempest-keypair-705526568", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNVRqI6LD4Y3pgiDulUQWtneu42jzDJXnWqkkjmTMo/s4lxfKxPGXfAoi73ybut3a4HDQ/RnQi8CU9SPXhFFAJuEs9nAgjES/+U5xaZ/HhcdKq97ndJGq27rGJ7I+sn5zA=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:20.846667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-6126ff58-0a8a-4d5b-98d7-faba413eb861 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:42:20.851246 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-6126ff58-0a8a-4d5b-98d7-faba413eb861 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:42:20.883682 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6126ff58-0a8a-4d5b-98d7-faba413eb861 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.056353 May 19 12:42:20.884726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1494/3109] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:42:20 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:21.550495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:21.551070 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:21.557425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:21.558039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:21.558605 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:21.559015 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:21.588011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-14db04c2-ff50-424f-876f-6df7077db722 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/dc745524-c0da-4062-b626-cd00d226360c" status: 200 len: 194 microversion: 2.1 time: 0.042247 May 19 12:42:21.588964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1616/3110] 10.222.0.35 () {58 vars in 1412 bytes} [Thu May 19 12:42:21 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/dc745524-c0da-4062-b626-cd00d226360c => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:21.599588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:21.600166 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:21.607238 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:21.607774 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:21.608597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:21.609146 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:21.689346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'services' on Instance uuid 2cb97d22-79d2-4528-9565-90cc111227b2 {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:42:21.768018 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-93270f55-c5b8-49ff-ac91-37f70c0b7508 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/dc745524-c0da-4062-b626-cd00d226360c" status: 202 len: 0 microversion: 2.1 time: 0.172719 May 19 12:42:21.768632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1495/3111] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:42:21 2022] DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments/dc745524-c0da-4062-b626-cd00d226360c => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:42:22.249442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1832431298", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f504b23c-8d0f-4020-83c4-474e12095c43"}], "security_groups": [{"name": "tempest-securitygroup--1634338261"}], "key_name": "tempest-keypair-705526568", "user_data": "}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:22.353667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] validate_networks() for [('f504b23c-8d0f-4020-83c4-474e12095c43', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:42:22.729594 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:42:22.730218 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:42:22.730510 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:42:22.730922 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:42:22.731327 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:42:22.741109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:22.741556 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:22.742855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:22.743290 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:22.760668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:42:22.765485 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:42:22.777770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:42:22.780042 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:42:22.835650 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-644c06b2-4d20-41da-84e0-1ff32c88382f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.590515 May 19 12:42:22.836717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1617/3112] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:42:22 2022] POST /compute/v2.1/servers => generated 402 bytes in 592 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:42:22.847854 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-82aaea3a-5c23-40c9-9c95-a55c7f8da1d6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:22.848549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-82aaea3a-5c23-40c9-9c95-a55c7f8da1d6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:22.915311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-82aaea3a-5c23-40c9-9c95-a55c7f8da1d6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:22.922674 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-82aaea3a-5c23-40c9-9c95-a55c7f8da1d6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1241 microversion: 2.1 time: 0.079535 May 19 12:42:22.923359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1496/3113] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:22 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1241 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:22.963642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:22.964631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:22.971642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:22.972121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:22.972972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:22.973457 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.002701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-595a16b2-062c-4645-8180-c70a54c9ac81 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043037 May 19 12:42:23.003147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1618/3114] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:42:22 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:42:23.120221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:23.121610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:23.128798 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.129247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.130016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.130457 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.158791 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0519bb18-52a3-40e7-9473-52640cdb40bd tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043246 May 19 12:42:23.159359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1497/3115] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:42:23 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:42:23.218364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:23.218867 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:23.225097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.225618 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.226450 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.226849 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.252361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:42:23.339056 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8c9a70b5-35e2-42fb-b84d-caf268e24c4a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 204 len: 0 microversion: 2.1 time: 0.124228 May 19 12:42:23.339696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1619/3116] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:42:23 2022] DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:42:23.348003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:23.348731 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:23.355828 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.356295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.357079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.357502 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.486095 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:23.496072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e7b4f460-9f03-4911-bc39-39c889305fa0 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1672 microversion: 2.1 time: 0.151704 May 19 12:42:23.497142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1498/3117] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:23 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1672 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:23.944778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:23.945912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:23.953859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.954382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:23.955262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:23.955769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:24.282686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:24.294601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-925f935a-57fc-4447-98c5-8049732d077d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1237 microversion: 2.1 time: 0.356544 May 19 12:42:24.295550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1620/3118] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:23 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1237 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:24.515651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:24.516284 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:24.523224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:24.523665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:24.524407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:24.524865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:24.662170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:24.669845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-51d0e0ac-8eff-4963-9317-66c9be8920f5 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1672 microversion: 2.1 time: 0.158134 May 19 12:42:24.670407 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1499/3119] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:24 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1672 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:25.318610 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:25.319254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:25.326798 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:25.327238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:25.327975 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:25.328393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:25.398511 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:25.406353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f387c3d4-d17c-479c-9b9a-a8070b61dda3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1293 microversion: 2.1 time: 0.093721 May 19 12:42:25.406918 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1621/3120] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:25 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1293 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:42:25.687599 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:25.688435 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:25.695463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:25.695991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:25.697032 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:25.697939 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:25.908810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:25.916978 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4c63efc1-0160-46b1-9f29-41761ccae96d tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1672 microversion: 2.1 time: 0.233257 May 19 12:42:25.917553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1500/3121] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:25 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1672 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:26.425637 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:26.426841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:26.434760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.435200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.435951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.436661 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.677058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:26.684868 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1053e2cc-3edd-4e35-8525-af6d15e6dc63 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.263123 May 19 12:42:26.685364 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1622/3122] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:26 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:26.829193 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-6a52c664-a7ce-45dd-aa0e-ae091a2e1ad5 req-636d7f19-9fa2-4408-a58b-d713e5e2d1e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cb97d22-79d2-4528-9565-90cc111227b2", "name": "network-vif-deleted", "tag": "e59a0d70-625c-49cb-b7bc-09a37ab5e0cf"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:26.842862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-6a52c664-a7ce-45dd-aa0e-ae091a2e1ad5 req-636d7f19-9fa2-4408-a58b-d713e5e2d1e4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.843713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-6a52c664-a7ce-45dd-aa0e-ae091a2e1ad5 req-636d7f19-9fa2-4408-a58b-d713e5e2d1e4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.855721 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-6a52c664-a7ce-45dd-aa0e-ae091a2e1ad5 req-636d7f19-9fa2-4408-a58b-d713e5e2d1e4 service nova] Creating event network-vif-deleted:e59a0d70-625c-49cb-b7bc-09a37ab5e0cf for instance 2cb97d22-79d2-4528-9565-90cc111227b2 on hv-cinder-80403 May 19 12:42:26.866035 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-6a52c664-a7ce-45dd-aa0e-ae091a2e1ad5 req-636d7f19-9fa2-4408-a58b-d713e5e2d1e4 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046091 May 19 12:42:26.867654 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1501/3123] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:42:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:26.937222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:26.937879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:26.952104 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.952320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.953122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.953746 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.987099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:26.987818 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:26.988435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lazy-loading 'fault' on Instance uuid 2cb97d22-79d2-4528-9565-90cc111227b2 {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:42:27.057885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eefde2311ec64430a6ed161238024df4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7917b551c3c4cc9be8a007b5cc987d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:27.068298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e93d300c-791b-4d6e-b0ee-adda347c53bf tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 200 len: 1327 microversion: 2.1 time: 0.135854 May 19 12:42:27.069095 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1623/3124] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:26 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 1327 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:27.369738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-f2200380-035f-41f5-8bf3-28ce452b101c req-fd13957c-3fae-40d9-acb0-fea0653c62d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aad342e1-3718-42b8-8416-e8e0c57db755", "tag": "ec6c2717-c229-476a-b476-422263810a55"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:27.383697 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-f2200380-035f-41f5-8bf3-28ce452b101c req-fd13957c-3fae-40d9-acb0-fea0653c62d5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:27.384570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-f2200380-035f-41f5-8bf3-28ce452b101c req-fd13957c-3fae-40d9-acb0-fea0653c62d5 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:27.404807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-f2200380-035f-41f5-8bf3-28ce452b101c req-fd13957c-3fae-40d9-acb0-fea0653c62d5 service nova] Creating event network-changed:ec6c2717-c229-476a-b476-422263810a55 for instance aad342e1-3718-42b8-8416-e8e0c57db755 on hv-cinder-80403 May 19 12:42:27.411916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-f2200380-035f-41f5-8bf3-28ce452b101c req-fd13957c-3fae-40d9-acb0-fea0653c62d5 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048109 May 19 12:42:27.413302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1502/3125] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:42:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:27.701755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:27.703019 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:27.709873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:27.710590 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:27.711696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:27.712295 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:27.919320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:27.928296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-335c74d3-ddd1-4ce6-adab-28b79c1333f2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.230371 May 19 12:42:27.928912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1624/3126] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:27 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:42:28.089311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:28.090299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:28.098920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:28.099513 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:28.100512 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:28.101132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:28.114910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 HTTP/1.1 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:28.116568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:28.118910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '2cb97d22-79d2-4528-9565-90cc111227b2'} May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:28.119965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:28.121844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:28.123427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:28.123427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0f87053c-ef12-4f07-830b-b161842d8e08 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 404 len: 111 microversion: 2.1 time: 0.035468 May 19 12:42:28.123427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1503/3127] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:28 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:28.129172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0a2ee292-38fa-444e-927e-c9b051f60de1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '.version_select at 0x7fe20c07a040>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:28.150073 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0a2ee292-38fa-444e-927e-c9b051f60de1 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1293115722" status: 202 len: 0 microversion: 2.1 time: 0.026496 May 19 12:42:28.150611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1625/3128] 10.222.0.35 () {58 vars in 1220 bytes} [Thu May 19 12:42:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1293115722 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:42:28.944456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:28.945419 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:28.951642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:28.952092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:28.952875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:28.953304 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:29.100844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:29.109053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-5dfe5dbc-71dd-4ef4-a3da-8ce5c05f750e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.168410 May 19 12:42:29.109613 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1504/3129] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:28 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:30.131331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:30.132213 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:30.142409 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:30.143135 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:30.144534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:30.145311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:30.282296 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:30.293227 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d23d83af-69cf-4a2d-a29a-aa7300c5abb3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.169278 May 19 12:42:30.293940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1626/3130] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:30 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:31.040372 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:31.040942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:31.047969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.048402 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.049199 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.049632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.063333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 HTTP/1.1 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:31.064951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.066755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '2cb97d22-79d2-4528-9565-90cc111227b2'} May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:31.067815 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.069560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:31.071259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:31.071259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-260a4745-b229-41af-bf46-f1827bba507f tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 404 len: 111 microversion: 2.1 time: 0.029830 May 19 12:42:31.071259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1505/3131] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:42:31 2022] DELETE /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:31.074755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:31.075367 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 2cb97d22-79d2-4528-9565-90cc111227b2] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:31.081796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.082242 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.082995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.083420 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.099036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 HTTP/1.1 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:42:31.100668 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.102378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '2cb97d22-79d2-4528-9565-90cc111227b2'} May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:31.103289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. May 19 12:42:31.105082 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:31.106577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 2cb97d22-79d2-4528-9565-90cc111227b2 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:31.106577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-b17bfd04-314e-4a3d-997d-d67d819a696a tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2" status: 404 len: 111 microversion: 2.1 time: 0.031181 May 19 12:42:31.106577 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1627/3132] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:31 2022] GET /compute/v2.1/servers/2cb97d22-79d2-4528-9565-90cc111227b2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:31.310839 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:31.311447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:31.317650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.318098 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.318852 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.319285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.453407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-290690f9-9236-4517-845b-a6845e178701 req-7537ce88-42eb-4317-b14f-77b5e1ad28c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aad342e1-3718-42b8-8416-e8e0c57db755", "name": "network-vif-plugged", "status": "completed", "tag": "ec6c2717-c229-476a-b476-422263810a55"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:31.461363 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:31.461745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-290690f9-9236-4517-845b-a6845e178701 req-7537ce88-42eb-4317-b14f-77b5e1ad28c1 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.462170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-290690f9-9236-4517-845b-a6845e178701 req-7537ce88-42eb-4317-b14f-77b5e1ad28c1 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.474013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-290690f9-9236-4517-845b-a6845e178701 req-7537ce88-42eb-4317-b14f-77b5e1ad28c1 service nova] Creating event network-vif-plugged:ec6c2717-c229-476a-b476-422263810a55 for instance aad342e1-3718-42b8-8416-e8e0c57db755 on hv-cinder-80403 May 19 12:42:31.474258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-99ef845e-3721-451a-9ced-2a6fc187719b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.167238 May 19 12:42:31.483961 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1506/3133] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:31 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:31.486122 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-290690f9-9236-4517-845b-a6845e178701 req-7537ce88-42eb-4317-b14f-77b5e1ad28c1 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036469 May 19 12:42:31.486760 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1628/3134] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:42:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:31.707459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:31.708438 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:31.714917 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.715368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.716117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.716691 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.792916 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f20f0ff9-0e17-47a1-ac60-74064c5763e8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action" status: 202 len: 0 microversion: 2.1 time: 0.089451 May 19 12:42:31.793548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1507/3135] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:42:31 2022] POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:42:31.802329 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:31.803025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:31.809629 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.810057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.810808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:31.811233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:31.960022 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:31.973079 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-56dc227a-4d63-48bd-9e2c-78401aee7179 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1751 microversion: 2.1 time: 0.174709 May 19 12:42:31.973873 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1629/3136] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:31 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1751 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:32.253191 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:32.253776 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:32.260413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.260888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.261649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.262070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.275823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 HTTP/1.1 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:32.277482 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.279331 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '4518b411-8fac-48f2-8826-7b91651826b1'} May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:32.280503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.282309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:32.284302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:32.284302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-225772b2-1cdb-41da-83be-409bda21f551 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 404 len: 111 microversion: 2.1 time: 0.030469 May 19 12:42:32.284302 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1508/3137] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:42:32 2022] DELETE /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:32.290432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:32.291337 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] [instance: 4518b411-8fac-48f2-8826-7b91651826b1] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:32.300399 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.301030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.302163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.302769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.317280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] HTTP exception thrown: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Request method failure captured: May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 HTTP/1.1 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjKNjg8y_R4QA2rz2uUi9aEwSxqpZRIlhYft85IxoUntidZYpQMlLalPUv6xMT0yjoJMrRTwyE3Q1yKvnnkyUanYFuBJHmVI1_2eec_lYc3wVPiTyeeoUApyWUgLOlx29r9M5nhg_8wPT3ax8AVtE0Jmi2FmVAUMz4CXiUs26VOW714 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:42:32.322043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c7917b551c3c4cc9be8a007b5cc987d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c7917b551c3c4cc9be8a007b5cc987d3"}]}] May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: c7917b551c3c4cc9be8a007b5cc987d3 May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441533884 May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: eefde2311ec64430a6ed161238024df4 May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeTestJSON-1441533884-project May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.323901 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '4518b411-8fac-48f2-8826-7b91651826b1'} May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:32.325000 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. May 19 12:42:32.326694 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:42:32.328253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] Returning 404 to user: Instance 4518b411-8fac-48f2-8826-7b91651826b1 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:32.328253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-64c49a84-5cd9-478f-9764-86bf13493871 tempest-AttachVolumeTestJSON-1441533884 tempest-AttachVolumeTestJSON-1441533884-project] 10.222.0.35 "GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1" status: 404 len: 111 microversion: 2.1 time: 0.037680 May 19 12:42:32.328253 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1630/3138] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:32 2022] GET /compute/v2.1/servers/4518b411-8fac-48f2-8826-7b91651826b1 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:32.491587 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:32.492404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:32.499134 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.499811 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.500717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:32.503508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:32.739110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:32.747995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-936258a4-0537-4eb4-97ca-cb1c0073f1ae tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1367 microversion: 2.1 time: 0.260284 May 19 12:42:32.757597 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1509/3139] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:32 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1367 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:32.991473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:32.992883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:32.996530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:32.997363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:33.005770 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.006151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:33.006928 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.007361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:33.027961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:33.154470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:33.162819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4b393810-d5ec-4385-8a42-5e20143300f7 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1751 microversion: 2.1 time: 0.175260 May 19 12:42:33.163429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1631/3140] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:32 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1751 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:33.764180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:33.765427 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:33.768479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:33.774257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.774726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:33.775490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.775899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:33.965288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:33.973282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4c6a32e8-c918-4869-9001-10eb34a1731f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1558 microversion: 2.1 time: 0.213228 May 19 12:42:33.973829 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1510/3141] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:33 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:33.989726 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:33.990216 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:33.997500 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.998136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:33.998686 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:33.999701 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:34.118767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-41b33206-21e4-43a0-8340-1df1a91b109f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.132843 May 19 12:42:34.125360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1632/3142] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:42:33 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/os-interface => generated 285 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:34.179968 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:34.180924 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:34.188040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:34.188519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:34.189324 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:34.189957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:34.362663 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:34.375942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3438f3b3-e754-4349-a764-a07f7bc60352 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.199174 May 19 12:42:34.377142 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1511/3143] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:34 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:35.394743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:35.395353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:35.402325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:35.402325 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:35.405657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:35.405657 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:35.782193 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:35.790426 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-22d0baf8-2a6a-4700-87fd-daf2e578b34a tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.400155 May 19 12:42:35.791004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1633/3144] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:35 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:35.985610 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-b8337389-c83f-4c94-9145-06c1f9d6ea63 req-38f14536-e4bd-4ea8-9c66-9e27e9120743 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aad342e1-3718-42b8-8416-e8e0c57db755", "tag": "ec6c2717-c229-476a-b476-422263810a55"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:35.995196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-b8337389-c83f-4c94-9145-06c1f9d6ea63 req-38f14536-e4bd-4ea8-9c66-9e27e9120743 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:35.995698 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-b8337389-c83f-4c94-9145-06c1f9d6ea63 req-38f14536-e4bd-4ea8-9c66-9e27e9120743 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:36.012886 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-b8337389-c83f-4c94-9145-06c1f9d6ea63 req-38f14536-e4bd-4ea8-9c66-9e27e9120743 service nova] Creating event network-changed:ec6c2717-c229-476a-b476-422263810a55 for instance aad342e1-3718-42b8-8416-e8e0c57db755 on hv-cinder-80403 May 19 12:42:36.018064 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-b8337389-c83f-4c94-9145-06c1f9d6ea63 req-38f14536-e4bd-4ea8-9c66-9e27e9120743 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036041 May 19 12:42:36.018903 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1512/3145] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:42:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:42:36.807779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:36.808393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:36.826883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:36.827428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:36.828289 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:36.829058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:36.995866 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:37.005522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-33019320-1561-45a9-9f96-94516b8d62ba tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.201710 May 19 12:42:37.006189 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1634/3146] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:36 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:38.019775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e511a7c7-92f4-4024-b07e-d9e52a34d032 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Action: 'create', calling method: .version_select at 0x7fe20c1cae50>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-2027822377", "metadata": {"Type": "work"}}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:38.022441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:38.023057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:38.029464 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:38.029910 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:38.030692 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e511a7c7-92f4-4024-b07e-d9e52a34d032 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 122 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.031016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-2027822377", "metadata": {"Type": "work"}}} May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.032834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-2027822377', 'metadata': {'Type': 'work'}}}} May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise error May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi On instance['volume']: May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi '#$%' May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.034026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 19 12:42:38.035986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 19 12:42:38.037631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 19 12:42:38.037631 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.037831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:38.037937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e511a7c7-92f4-4024-b07e-d9e52a34d032 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.037937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e511a7c7-92f4-4024-b07e-d9e52a34d032 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.146272 May 19 12:42:38.037937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1513/3147] 10.222.0.35 () {60 vars in 1155 bytes} [Thu May 19 12:42:37 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 148 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:42:38.042376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bbfe23b1-9565-4d01-8414-9273f9b67783 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Action: 'create', calling method: .version_select at 0x7fe1fff974c0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1312564874", "metadata": {"Type": "work"}}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bbfe23b1-9565-4d01-8414-9273f9b67783 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 120 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.044683 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1312564874", "metadata": {"Type": "work"}}} May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.046455 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1312564874', 'metadata': {'Type': 'work'}}}} May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise error May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi On instance['volume']: May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi '0' May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.047560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 19 12:42:38.050168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 19 12:42:38.051732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 19 12:42:38.051732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.051732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-bbfe23b1-9565-4d01-8414-9273f9b67783 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.051732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-bbfe23b1-9565-4d01-8414-9273f9b67783 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.007454 May 19 12:42:38.051732 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1514/3148] 10.222.0.35 () {60 vars in 1155 bytes} [Thu May 19 12:42:38 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 19 12:42:38.055957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ee569b1-8352-449d-a16c-7f90b9b71cfe tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Action: 'create', calling method: .version_select at 0x7fe20c191820>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1897352580", "metadata": {"Type": "work"}}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ee569b1-8352-449d-a16c-7f90b9b71cfe tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 119 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.058282 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1897352580", "metadata": {"Type": "work"}}} May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.059929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1897352580', 'metadata': {'Type': 'work'}}}} May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise error May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi On instance['volume']: May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi '' May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.061278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 19 12:42:38.063449 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 19 12:42:38.065862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 19 12:42:38.065862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.065862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3ee569b1-8352-449d-a16c-7f90b9b71cfe tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.065862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3ee569b1-8352-449d-a16c-7f90b9b71cfe tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.007094 May 19 12:42:38.065862 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1515/3149] 10.222.0.35 () {60 vars in 1155 bytes} [Thu May 19 12:42:38 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:42:38.072462 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-854d74aa-4e1e-4361-9440-0291165633a4 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Calling method '.version_select at 0x7fe20c1448b0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:38.104221 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-854d74aa-4e1e-4361-9440-0291165633a4 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] HTTP exception thrown: Volume tempest-invalid-128267598 could not be found. May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-854d74aa-4e1e-4361-9440-0291165633a4 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-128267598 HTTP/1.1 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.107503 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Volume tempest-invalid-128267598 could not be found. May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'tempest-invalid-128267598'} May 19 12:42:38.109329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-128267598 could not be found. (HTTP 404) (Request-ID: req-c3b8d864-fd3e-450f-9139-9462b1cc9749) May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.110560 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:42:38.112249 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-128267598 could not be found. May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:38.113857 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-128267598 could not be found. May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-128267598 could not be found. May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-854d74aa-4e1e-4361-9440-0291165633a4 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 404 to user: Volume tempest-invalid-128267598 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-854d74aa-4e1e-4361-9440-0291165633a4 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/tempest-invalid-128267598" status: 404 len: 98 microversion: 2.1 time: 0.041950 May 19 12:42:38.115760 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1516/3150] 10.222.0.35 () {58 vars in 1214 bytes} [Thu May 19 12:42:38 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-128267598 => generated 98 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) May 19 12:42:38.121305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-c071c579-f164-42f4-ae2d-643552120b15 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005653 May 19 12:42:38.122074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1517/3151] 10.222.0.35 () {58 vars in 1139 bytes} [Thu May 19 12:42:38 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) May 19 12:42:38.130274 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8864043b-db39-4cf1-b6d9-35c940b18b8e tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003481 May 19 12:42:38.131294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1518/3152] 10.222.0.35 () {58 vars in 1136 bytes} [Thu May 19 12:42:38 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 19 12:42:38.139055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1918fb14-4536-4340-96fc-b29bc237b220 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Calling method '.version_select at 0x7fe1fff97940>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:38.166822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-1918fb14-4536-4340-96fc-b29bc237b220 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] HTTP exception thrown: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1918fb14-4536-4340-96fc-b29bc237b220 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/os-volumes/a01a9508-41de-4cdd-9a11-af7a58fc435b HTTP/1.1 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.168808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'a01a9508-41de-4cdd-9a11-af7a58fc435b'} May 19 12:42:38.170633 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. (HTTP 404) (Request-ID: req-9cbf36b3-ebec-4c9e-a973-c93ddac01098) May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.171882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 19 12:42:38.174718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 19 12:42:38.177377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1918fb14-4536-4340-96fc-b29bc237b220 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 404 to user: Volume a01a9508-41de-4cdd-9a11-af7a58fc435b could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1918fb14-4536-4340-96fc-b29bc237b220 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/a01a9508-41de-4cdd-9a11-af7a58fc435b" status: 404 len: 109 microversion: 2.1 time: 0.034656 May 19 12:42:38.179810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1519/3153] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:42:38 2022] DELETE /compute/v2.1/os-volumes/a01a9508-41de-4cdd-9a11-af7a58fc435b => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 19 12:42:38.181632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b945e5ce-d199-4146-9cf4-f7c2cf699898 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Calling method '.version_select at 0x7fe20c175c10>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:38.190307 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:38.199428 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6afedec9-294a-4759-bb4c-2c2e523197c8 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.180773 May 19 12:42:38.200003 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1635/3154] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:38 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:38.206992 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-b945e5ce-d199-4146-9cf4-f7c2cf699898 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] HTTP exception thrown: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b945e5ce-d199-4146-9cf4-f7c2cf699898 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Request method failure captured: May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/os-volumes/c350e3e5-f326-4cc4-8f40-abc98c031637 HTTP/1.1 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjs9y0fJ21LIrOt4anCvLfUL3cRLQgEnts_4Ejy08x_eWbqIk4by62-40s_BQnhKMVyAp3RUdSZ_hOfNKtT-MnW6_LZHkKWOyssOMCmwZMPlekR1gjS0YRaqEs3Qa97SOhfe2ZC-ebLE8XsAe4W7ayV_SEeK-UpIVZYjG6i-4wKVCOA May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:42:38.208967 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/58194a397ea646629fe8e4a725dea646"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_58194a397ea646629fe8e4a725dea646"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/58194a397ea646629fe8e4a725dea646"}]}] May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 58194a397ea646629fe8e4a725dea646 May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesNegativeTest-1045759784 May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: b127a82a988d4eddb9840303ac092003 May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesNegativeTest-1045759784-project May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'c350e3e5-f326-4cc4-8f40-abc98c031637'} May 19 12:42:38.210846 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. (HTTP 404) (Request-ID: req-ede0caba-b608-420b-b76d-d77af9d8842a) May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.211918 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi item = cinderclient( May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 19 12:42:38.213540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 19 12:42:38.215068 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-b945e5ce-d199-4146-9cf4-f7c2cf699898 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] Returning 404 to user: Volume c350e3e5-f326-4cc4-8f40-abc98c031637 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-b945e5ce-d199-4146-9cf4-f7c2cf699898 tempest-VolumesNegativeTest-1045759784 tempest-VolumesNegativeTest-1045759784-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/c350e3e5-f326-4cc4-8f40-abc98c031637" status: 404 len: 109 microversion: 2.1 time: 0.034528 May 19 12:42:38.216676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1520/3155] 10.222.0.35 () {58 vars in 1244 bytes} [Thu May 19 12:42:38 2022] GET /compute/v2.1/os-volumes/c350e3e5-f326-4cc4-8f40-abc98c031637 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:42:39.216240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:39.216880 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:39.223387 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:39.223827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:39.224571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:39.225018 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:39.381743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:39.390529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-264024d7-22e5-48ea-9cfe-62c7b1491767 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.178042 May 19 12:42:39.391175 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1636/3156] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:39 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:40.408415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:40.409474 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:40.416536 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:40.417041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:40.417820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:40.418226 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:40.580292 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:40.588558 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e8b12d0c-d1dc-4f2b-b15d-b040059eac88 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.184523 May 19 12:42:40.589174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1521/3157] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:40 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:41.606207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:41.606815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:41.613598 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:41.614038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:41.614808 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:41.615201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:41.957250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:41.965269 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-60421647-1928-4853-a9d0-9e544329a867 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.362949 May 19 12:42:41.965838 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1637/3158] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:41 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:42.982716 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:42.983415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:42.990585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:42.991003 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:42.992126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:42.992577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:42.995328 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:43.206112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:43.214628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28848980-b0c3-494c-93d4-97f8b14e7364 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.235791 May 19 12:42:43.215201 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1522/3159] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:42 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:44.236094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:44.237183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:44.241043 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=3,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:44.247817 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:44.248262 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:44.249046 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:44.249477 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:44.406324 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:44.418584 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c75542ec-d529-4f7c-9abb-ed17630a51f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.188793 May 19 12:42:44.419194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1638/3160] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:44 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:45.440425 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:45.441470 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:45.445161 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:45.451210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:45.451651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:45.452403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:45.452865 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:45.599664 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:45.608228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae09429c-b2b4-48c9-baf9-47fd25b813f0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.174544 May 19 12:42:45.608799 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1523/3161] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:45 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:46.629651 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:46.630540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:46.642106 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:46.642809 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:46.644053 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:46.644814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:46.790915 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:46.803270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6405147d-6387-4401-aa63-6b9d216ca997 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.180149 May 19 12:42:46.804226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1639/3162] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:46 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:47.821170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:47.821725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:47.828609 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:47.829070 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:47.829817 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:47.830245 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:48.005346 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:48.013644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-2a9cf144-ce97-488a-ab89-eb20a283c542 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.196464 May 19 12:42:48.016675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1524/3163] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:47 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:49.034871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:49.035875 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:49.046999 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:49.047737 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:49.049017 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:49.049721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:49.200380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:49.208773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-086613fe-517f-4bc8-8485-394fc46e2d81 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.180242 May 19 12:42:49.209344 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1640/3164] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:49 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:50.230471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:50.231090 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:50.238051 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:50.238574 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:50.239494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:50.239995 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:50.398168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:50.406764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cc078eae-3a55-4d95-b58f-bc7537710694 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1756 microversion: 2.1 time: 0.180366 May 19 12:42:50.407336 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1525/3165] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:50 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:42:51.425815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:42:51.426583 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:42:51.433271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:51.433716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:51.434473 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:42:51.435199 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:42:51.636749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:42:51.646009 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-916f3dca-f14e-4164-87df-cc169a9810a5 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1759 microversion: 2.1 time: 0.225515 May 19 12:42:51.646719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1641/3166] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:42:51 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1759 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:12.204822 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:12.205408 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:12.207847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=5 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:12.208617 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=23 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:12.213757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.214192 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.214942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.215358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.239789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-522eaffa-be27-4066-9563-3247079985ce tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/ips" status: 200 len: 150 microversion: 2.1 time: 0.038814 May 19 12:43:12.240305 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1526/3167] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:43:12 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/ips => generated 150 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:12.252407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "428a739e-a87d-4756-ac58-ad45c67b7f37"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:12.254515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:12.257105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=7 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:12.258620 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=60 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:12.265179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.265648 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.266434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.266862 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.351356 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: POST /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/os-volume_attachments HTTP/1.1 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 74 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:43:12.353048 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {"volumeAttachment": {"volumeId": "428a739e-a87d-4756-ac58-ad45c67b7f37"}} May 19 12:43:12.354951 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': 'aad342e1-3718-42b8-8416-e8e0c57db755', 'body': {'volumeAttachment': {'volumeId': '428a739e-a87d-4756-ac58-ad45c67b7f37'}}} May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:12.356055 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:12.357898 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 400 to user: Invalid volume: volume 428a739e-a87d-4756-ac58-ad45c67b7f37 is already attached to instances: 4440209e-114a-4e00-866c-006bba2d62fa {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3b77f30f-5add-411d-a8b5-b25b8b754dc6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.107280 May 19 12:43:12.359685 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1642/3168] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:43:12 2022] POST /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755/os-volume_attachments => generated 174 bytes in 108 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 19 12:43:12.364195 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:12.364718 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:12.371422 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.371875 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.372848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.373272 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.399722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:43:12.479761 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-330ca3b3-9b93-4cef-a305-831d119c5adb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 204 len: 0 microversion: 2.1 time: 0.118940 May 19 12:43:12.481106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1527/3169] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:43:12 2022] DELETE /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:43:12.489201 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:12.489869 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:12.496240 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.496699 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.498495 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:12.498924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:12.630486 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:12.638900 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6c2d9a41-3743-4733-ba54-148041e4fbdc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1682 microversion: 2.1 time: 0.153590 May 19 12:43:12.639470 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1643/3170] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:12 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:13.657262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:13.658657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:13.671648 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:13.671915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:13.672807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:13.673316 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:13.807011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:13.814632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-600ba39f-f4e8-484c-ab67-fbf1feae7807 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1682 microversion: 2.1 time: 0.162071 May 19 12:43:13.815268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1528/3171] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:13 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1682 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:14.831467 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:14.832090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:14.838775 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:14.839217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:14.839964 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:14.840388 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:15.063727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:15.072482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0c848499-f185-42fa-b685-e734a29c05c8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 200 len: 1682 microversion: 2.1 time: 0.244840 May 19 12:43:15.073207 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1644/3172] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:14 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 1682 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:15.167738 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1763642697", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "networks": [{"uuid": "6bbeebae-8969-43b5-b49f-81ba4270e2d1"}]}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:15.219135 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] validate_networks() for [('6bbeebae-8969-43b5-b49f-81ba4270e2d1', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:43:15.338957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-2265c7e5-c963-4935-9b71-efd5bebc7ad3 req-65878a5f-6b03-45ad-ba87-0390dafa32a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aad342e1-3718-42b8-8416-e8e0c57db755", "name": "network-vif-deleted", "tag": "ec6c2717-c229-476a-b476-422263810a55"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:15.350807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2265c7e5-c963-4935-9b71-efd5bebc7ad3 req-65878a5f-6b03-45ad-ba87-0390dafa32a0 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:15.351225 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-2265c7e5-c963-4935-9b71-efd5bebc7ad3 req-65878a5f-6b03-45ad-ba87-0390dafa32a0 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:15.365060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-2265c7e5-c963-4935-9b71-efd5bebc7ad3 req-65878a5f-6b03-45ad-ba87-0390dafa32a0 service nova] Creating event network-vif-deleted:ec6c2717-c229-476a-b476-422263810a55 for instance aad342e1-3718-42b8-8416-e8e0c57db755 on hv-cinder-80403 May 19 12:43:15.380820 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-2265c7e5-c963-4935-9b71-efd5bebc7ad3 req-65878a5f-6b03-45ad-ba87-0390dafa32a0 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046589 May 19 12:43:15.384460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1645/3173] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:15.624561 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:43:15.625101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:43:15.625629 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:43:15.626124 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:43:15.626589 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:43:15.635484 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:15.635989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:15.636872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:15.637368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:15.654388 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Getting quotas for project cedf35fd585743f2893b58a2fda9edff. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:43:15.658384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Getting quotas for user 912741d6d24b42d5bf98cf0d2cc8e43d and project cedf35fd585743f2893b58a2fda9edff. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:43:15.669486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:43:15.671754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:43:15.717224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-56944a72-6ca2-42dc-9a77-b9c50a276fca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.553333 May 19 12:43:15.718112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1529/3174] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:43:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 555 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 19 12:43:15.728734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-55b03304-7ee9-4fad-b646-7fb8667d09a5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:15.729423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-55b03304-7ee9-4fad-b646-7fb8667d09a5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:15.795241 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-55b03304-7ee9-4fad-b646-7fb8667d09a5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:15.802155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-55b03304-7ee9-4fad-b646-7fb8667d09a5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1214 microversion: 2.1 time: 0.077881 May 19 12:43:15.802735 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1646/3175] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:15 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1214 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:16.090011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:16.090860 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:16.097964 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.098539 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.099548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.100380 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.116628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 HTTP/1.1 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:43:16.120403 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:43:16.122326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'aad342e1-3718-42b8-8416-e8e0c57db755'} May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:16.123614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:43:16.125804 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:16.127568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:16.127568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7122cdff-7a83-4a30-9cc4-165acfdf5f4e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 404 len: 111 microversion: 2.1 time: 0.039096 May 19 12:43:16.127568 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1530/3176] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:16 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:43:16.135285 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8a3dc6dc-99ec-4e51-b09c-b9213b145183 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '.version_select at 0x7fe20cc12700>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:16.155349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8a3dc6dc-99ec-4e51-b09c-b9213b145183 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-705526568" status: 202 len: 0 microversion: 2.1 time: 0.024553 May 19 12:43:16.155940 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1647/3177] 10.222.0.35 () {58 vars in 1217 bytes} [Thu May 19 12:43:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-705526568 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:43:16.822062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:16.823504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:16.831089 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.831773 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.832779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.833368 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.868320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:16.869957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:16.877441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.878164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.879149 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:16.880033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:16.922600 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:16.934183 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-19bc1724-3247-4f09-b8ad-bc312a8d1731 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1274 microversion: 2.1 time: 0.118020 May 19 12:43:16.934884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1531/3178] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:16 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1274 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:16.987697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lazy-loading 'services' on Instance uuid 4440209e-114a-4e00-866c-006bba2d62fa {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:17.061529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1e3f2aed-d577-41f1-b961-1b12f8f70c6d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments/428a739e-a87d-4756-ac58-ad45c67b7f37" status: 202 len: 0 microversion: 2.1 time: 0.200466 May 19 12:43:17.062133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1648/3179] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:43:16 2022] DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments/428a739e-a87d-4756-ac58-ad45c67b7f37 => generated 0 bytes in 203 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:43:17.070929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:17.071844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:17.078256 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:17.078701 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:17.079467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:17.079899 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:17.110810 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-cfd15e18-9ec2-499e-b43a-3ec69829bbe2 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044075 May 19 12:43:17.114823 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1532/3180] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:43:17 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:17.952505 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:17.953229 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:17.960300 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:17.960781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:17.961616 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:17.962025 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.123723 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:18.124667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:18.131590 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.132133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.132894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.133320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.162075 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-77920962-8361-4e7c-a056-be966fd915ea tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042469 May 19 12:43:18.162644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1533/3181] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:43:18 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:43:18.182052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:18.189961 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c5e68c26-53f7-4ed5-aa58-ccb3a2f287b7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.242091 May 19 12:43:18.190530 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1649/3182] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:17 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:18.222213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:18.222750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:18.229364 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.229812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.230611 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.231044 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.258050 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:43:18.302762 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-0b85cb2d-a485-4b5c-b2d7-d1f93d6e6c35 req-953150e4-fa0b-46e3-93da-b6adcd42de25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4330061e-acfa-4968-8c84-3e3a7a6f4bba", "tag": "7d420681-eaa8-4d29-ba49-0e43ac3d3449"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:18.314924 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-0b85cb2d-a485-4b5c-b2d7-d1f93d6e6c35 req-953150e4-fa0b-46e3-93da-b6adcd42de25 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.315462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-0b85cb2d-a485-4b5c-b2d7-d1f93d6e6c35 req-953150e4-fa0b-46e3-93da-b6adcd42de25 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.332188 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-0b85cb2d-a485-4b5c-b2d7-d1f93d6e6c35 req-953150e4-fa0b-46e3-93da-b6adcd42de25 service nova] Creating event network-changed:7d420681-eaa8-4d29-ba49-0e43ac3d3449 for instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba on hv-cinder-80403 May 19 12:43:18.337819 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-0b85cb2d-a485-4b5c-b2d7-d1f93d6e6c35 req-953150e4-fa0b-46e3-93da-b6adcd42de25 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038757 May 19 12:43:18.338647 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1650/3183] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:18.346099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d58ec150-e075-4b5b-a5ea-fa926a597cdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 204 len: 0 microversion: 2.1 time: 0.127221 May 19 12:43:18.350393 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1534/3184] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:43:18 2022] DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:43:18.356280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:18.357008 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:18.363361 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.363796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.364538 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:18.364981 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:18.499147 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:18.507187 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-46b7fdb8-020f-48ad-bbcf-36d6dba1b1ca tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1681 microversion: 2.1 time: 0.155381 May 19 12:43:18.507859 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1651/3185] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:18 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:19.215812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:19.216989 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:19.227688 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:19.228133 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:19.228923 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:19.229357 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:19.384778 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:19.392385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1645b8f2-7936-4172-a39f-c5c99fbf2e80 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.187310 May 19 12:43:19.392991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1535/3186] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:19 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:19.526124 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:19.526743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:19.533698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:19.534137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:19.534881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:19.535308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:19.680205 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:19.687971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-23e2f802-e3eb-49d2-8038-59686ce52bdf tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1681 microversion: 2.1 time: 0.166413 May 19 12:43:19.688550 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1652/3187] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:19 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1681 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:20.409695 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:20.410935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:20.417559 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:20.417999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:20.418746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:20.419174 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:20.643405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:20.651180 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-90561503-90dd-48c5-b195-5974e582f1f9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.245288 May 19 12:43:20.651741 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1536/3188] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:20 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:20.705680 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:20.706363 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:20.713011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:20.713432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:20.714468 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:20.715700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:20.779904 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:20.787788 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-0f0459d4-5bf5-4896-8b6c-884673b792dc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1615 microversion: 2.1 time: 0.086001 May 19 12:43:20.788509 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1653/3189] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:20 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1615 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:21.067696 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-ccf16e28-b269-42f5-8741-50e10ae2b14d req-a8b78caa-68a5-4523-950c-fddaa8192d86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4440209e-114a-4e00-866c-006bba2d62fa", "name": "network-vif-deleted", "tag": "17d404c2-5bde-4e15-8153-61e1db648429"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:21.075638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-ccf16e28-b269-42f5-8741-50e10ae2b14d req-a8b78caa-68a5-4523-950c-fddaa8192d86 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.076069 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-ccf16e28-b269-42f5-8741-50e10ae2b14d req-a8b78caa-68a5-4523-950c-fddaa8192d86 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.087280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-ccf16e28-b269-42f5-8741-50e10ae2b14d req-a8b78caa-68a5-4523-950c-fddaa8192d86 service nova] Creating event network-vif-deleted:17d404c2-5bde-4e15-8153-61e1db648429 for instance 4440209e-114a-4e00-866c-006bba2d62fa on hv-cinder-80403 May 19 12:43:21.092233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-ccf16e28-b269-42f5-8741-50e10ae2b14d req-a8b78caa-68a5-4523-950c-fddaa8192d86 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028554 May 19 12:43:21.092883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1537/3190] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:21.667452 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:21.668059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:21.674462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.674956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.675725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.676155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.806322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:21.807297 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:21.818413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.822038 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.823005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.823467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.826742 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:21.835443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-bcaf2a34-b26d-4948-aec0-c6cbb41440c3 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.171786 May 19 12:43:21.836611 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1654/3191] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:21 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:21.865730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:21.866492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:21.867065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lazy-loading 'fault' on Instance uuid 4440209e-114a-4e00-866c-006bba2d62fa {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:21.921781 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:21.929362 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8d717444-d172-4962-b365-4966971d9f67 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 200 len: 1332 microversion: 2.1 time: 0.127763 May 19 12:43:21.930176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1538/3192] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:21 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 1332 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:22.855120 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:22.855896 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:22.865097 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:22.865683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:22.866997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:22.867561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:22.947545 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:22.948182 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:22.955186 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:22.955786 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:22.956577 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:22.957008 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:22.970844 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa HTTP/1.1 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:43:22.972538 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:43:22.974171 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '4440209e-114a-4e00-866c-006bba2d62fa'} May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:22.975082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:43:22.976659 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:22.978054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:22.978054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-73055015-b3a6-46a3-a5f4-9bfc5700a0cb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 404 len: 111 microversion: 2.1 time: 0.030390 May 19 12:43:22.978054 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1539/3193] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:22 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:43:22.983390 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e6ad9781-194c-46ae-8e89-8570e36b815d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '.version_select at 0x7fe1fff750d0>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:23.002779 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e6ad9781-194c-46ae-8e89-8570e36b815d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1850550406" status: 202 len: 0 microversion: 2.1 time: 0.022950 May 19 12:43:23.003353 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1540/3194] 10.222.0.35 () {58 vars in 1220 bytes} [Thu May 19 12:43:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1850550406 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:43:23.004210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:23.011949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3ba425c0-b9dc-4fa3-8c6f-fe4fa6ec0cc4 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.162453 May 19 12:43:23.012527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1655/3195] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:22 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:23.624808 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ce43f857-1ec3-47f0-b8db-3cb402796dba tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: .version_select at 0x7fe20c3c68b0>, body: {"keypair": {"name": "tempest-keypair-653431057", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK0T4H4NnJpWXcO6dz4hljDAL1mhaqO8N5Wul/lIIi6DLo1y4d23Qd1U9UehsbBDL/geG3wpmM7P7RH9wrJ8AnZ9JwROlh/cYjVyRUMTVd/8K++c66MSn2GXa4T/kFnR9Q=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:23.639550 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-ce43f857-1ec3-47f0-b8db-3cb402796dba tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:43:23.643622 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-ce43f857-1ec3-47f0-b8db-3cb402796dba tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:43:23.666208 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ce43f857-1ec3-47f0-b8db-3cb402796dba tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045398 May 19 12:43:23.666807 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1541/3196] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:43:23 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:24.029593 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:24.031047 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:24.038180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:24.038878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:24.039841 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:24.040448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:24.175453 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:24.183434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d5cfbfec-1297-40ca-a57a-684fd23ce0d7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.158377 May 19 12:43:24.183984 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1656/3197] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:24 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:25.035157 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1708665175", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f504b23c-8d0f-4020-83c4-474e12095c43"}], "security_groups": [{"name": "tempest-securitygroup--2128042769"}], "key_name": "tempest-keypair-653431057", "user_data": "}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:25.140476 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] validate_networks() for [('f504b23c-8d0f-4020-83c4-474e12095c43', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:43:25.200790 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:25.201368 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:25.208255 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:25.208728 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:25.209484 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:25.209878 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:25.339323 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:25.346965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f94694fa-9819-4a65-8d50-64a19060dedb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.150147 May 19 12:43:25.347801 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1657/3198] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:25 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:25.515527 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:43:25.516011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:43:25.516569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:43:25.517067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:43:25.517665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:43:25.526588 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:25.527026 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:25.527830 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:25.528250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:25.548593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:43:25.552800 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:43:25.563915 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:43:25.566523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:43:25.614700 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3390c5e9-6e26-4bb9-afd5-08c3d6058fe6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.584268 May 19 12:43:25.615522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1542/3199] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:43:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:43:25.644145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c0e2722f-0dd2-4898-952b-562f29907917 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:25.645161 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c0e2722f-0dd2-4898-952b-562f29907917 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:25.662623 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-e6e9d2cf-7b00-453f-a03f-1870ae5d657f req-a3036a08-9999-4820-aafd-f4285e80405d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4330061e-acfa-4968-8c84-3e3a7a6f4bba", "name": "network-vif-plugged", "status": "completed", "tag": "7d420681-eaa8-4d29-ba49-0e43ac3d3449"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:25.686188 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6e9d2cf-7b00-453f-a03f-1870ae5d657f req-a3036a08-9999-4820-aafd-f4285e80405d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:25.686935 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-e6e9d2cf-7b00-453f-a03f-1870ae5d657f req-a3036a08-9999-4820-aafd-f4285e80405d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:25.709094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-e6e9d2cf-7b00-453f-a03f-1870ae5d657f req-a3036a08-9999-4820-aafd-f4285e80405d service nova] Creating event network-vif-plugged:7d420681-eaa8-4d29-ba49-0e43ac3d3449 for instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba on hv-cinder-80403 May 19 12:43:25.718461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-e6e9d2cf-7b00-453f-a03f-1870ae5d657f req-a3036a08-9999-4820-aafd-f4285e80405d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062423 May 19 12:43:25.720103 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1543/3200] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:25.752435 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c0e2722f-0dd2-4898-952b-562f29907917 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:25.765990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c0e2722f-0dd2-4898-952b-562f29907917 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1241 microversion: 2.1 time: 0.127491 May 19 12:43:25.766811 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1658/3201] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:25 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1241 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:26.364564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:26.365343 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:26.372530 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:26.373283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:26.374237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:26.374554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:26.509495 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:26.517312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7a414df9-106b-4f35-8071-6ae73c576484 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1314 microversion: 2.1 time: 0.157399 May 19 12:43:26.517883 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1544/3202] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:26 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:26.783319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:26.784990 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:26.792121 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:26.792566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:26.793345 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:26.793777 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:26.864160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:26.872183 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c50e5c0b-173c-4e76-865f-7b09972bdedc tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1293 microversion: 2.1 time: 0.092953 May 19 12:43:26.872786 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1659/3203] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:26 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1293 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:27.535657 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:27.536257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:27.543325 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:27.543767 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:27.544504 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:27.544955 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:27.549013 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:27.684602 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:27.692467 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d8eff9ca-f822-4d45-8ab7-cb816814afd8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1499 microversion: 2.1 time: 0.160692 May 19 12:43:27.693056 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1545/3204] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:27 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1499 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:27.889673 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:27.890319 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:27.899023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:27.899921 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:27.900893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:27.901769 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:27.975182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:27.984957 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-71337f71-f0ba-4a8e-ba95-c4a450c1d169 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1311 microversion: 2.1 time: 0.099317 May 19 12:43:27.992758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1660/3205] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:27 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1311 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:28.227237 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:28.227769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:28.234826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:28.235528 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:28.236168 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:28.236596 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:28.261618 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:43:28.329671 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=90,nova_api:UPDATE=2,nova_api:INSERT=7,nova_api:DELETE=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:28.349360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-edce0452-393d-4605-ba0f-26a9d8a875e6 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 204 len: 0 microversion: 2.1 time: 0.125614 May 19 12:43:28.350253 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1546/3206] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:43:28 2022] DELETE /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:43:28.359338 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:28.359960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:28.367002 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:28.367459 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:28.368408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:28.368864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:28.495372 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:28.510727 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7d6817df-b51c-47ea-ab7d-99349baa748e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1505 microversion: 2.1 time: 0.155445 May 19 12:43:28.511515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1661/3207] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:28 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1505 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:29.011882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:29.013033 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:29.024565 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:29.025385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:29.026627 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:29.027396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:29.164224 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:29.172111 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4df1894a-f822-402d-b84e-42a9e505f436 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.167444 May 19 12:43:29.172670 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1547/3208] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:29 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:29.528490 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:29.529222 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:29.536132 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:29.536570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:29.537374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:29.537802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:29.542244 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:29.686836 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:29.695380 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6420f6ef-3e38-4074-9b00-1032b68ac552 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1505 microversion: 2.1 time: 0.170613 May 19 12:43:29.705232 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1662/3209] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:29 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1505 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:30.198285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:30.199400 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:30.202111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1185963528"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:30.202985 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:30.207881 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.208337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.208802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.209137 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.209569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.209781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.210840 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.211423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.438650 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:30.448628 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-842e6b4d-2da4-40c5-9b9a-dc4f0d9e25ab tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.263637 May 19 12:43:30.457893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1548/3210] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:30 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:30.465119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-7a324ab1-daed-4f35-8710-5003d5888780 req-5a8c90b9-1854-4210-b9ff-db343f2bccbc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4330061e-acfa-4968-8c84-3e3a7a6f4bba", "name": "network-vif-deleted", "tag": "7d420681-eaa8-4d29-ba49-0e43ac3d3449"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:30.476562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-7a324ab1-daed-4f35-8710-5003d5888780 req-5a8c90b9-1854-4210-b9ff-db343f2bccbc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.477048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-7a324ab1-daed-4f35-8710-5003d5888780 req-5a8c90b9-1854-4210-b9ff-db343f2bccbc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.489094 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-7a324ab1-daed-4f35-8710-5003d5888780 req-5a8c90b9-1854-4210-b9ff-db343f2bccbc service nova] Creating event network-vif-deleted:7d420681-eaa8-4d29-ba49-0e43ac3d3449 for instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba on hv-cinder-80403 May 19 12:43:30.500546 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-7a324ab1-daed-4f35-8710-5003d5888780 req-5a8c90b9-1854-4210-b9ff-db343f2bccbc service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049974 May 19 12:43:30.501463 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1549/3211] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:30.515053 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-4d4c557a-a7bc-47eb-aaa2-9eb08ab10f84 req-1f2e163f-0400-48ff-af8b-1163770d38ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7b74f125-b188-4751-baed-8d0bb14ded3b", "tag": "877681eb-a6ae-4d76-8be0-e607037850b1"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:30.527428 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-4d4c557a-a7bc-47eb-aaa2-9eb08ab10f84 req-1f2e163f-0400-48ff-af8b-1163770d38ce service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.528255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-4d4c557a-a7bc-47eb-aaa2-9eb08ab10f84 req-1f2e163f-0400-48ff-af8b-1163770d38ce service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.528826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.network.security_group_api [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Removing security group 1c5346d4-b915-4fd9-aeb2-38ba58129fd8 from port fedd35b6-9b61-414c-b8f9-173f39f64c6e May 19 12:43:30.541805 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-4d4c557a-a7bc-47eb-aaa2-9eb08ab10f84 req-1f2e163f-0400-48ff-af8b-1163770d38ce service nova] Creating event network-changed:877681eb-a6ae-4d76-8be0-e607037850b1 for instance 7b74f125-b188-4751-baed-8d0bb14ded3b on hv-cinder-80403 May 19 12:43:30.546952 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-4d4c557a-a7bc-47eb-aaa2-9eb08ab10f84 req-1f2e163f-0400-48ff-af8b-1163770d38ce service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041694 May 19 12:43:30.547726 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1550/3212] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:30.713126 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:30.713816 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:30.721421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.722205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.722981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.723910 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.757110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:30.757583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:30.758029 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lazy-loading 'fault' on Instance uuid 4330061e-acfa-4968-8c84-3e3a7a6f4bba {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:30.798845 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:30.807067 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ca751be2-2392-4088-955e-4ea390e66dd5 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 200 len: 1304 microversion: 2.1 time: 0.098266 May 19 12:43:30.807667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1551/3213] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:30 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 1304 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:30.933977 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d0abe103-fefc-4c5a-aa86-e59474212770 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action" status: 202 len: 0 microversion: 2.1 time: 0.734546 May 19 12:43:30.938389 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1663/3214] 10.222.0.35 () {60 vars in 1277 bytes} [Thu May 19 12:43:30 2022] POST /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/action => generated 0 bytes in 739 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 19 12:43:31.465780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:31.467030 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:31.473957 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:31.474413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:31.475177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:31.475614 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:31.613048 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:31.621722 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d49cc214-4e58-4673-9cb3-ac0059c0aad4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.159726 May 19 12:43:31.622319 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1552/3215] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:31 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:31.824164 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:31.824827 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 4330061e-acfa-4968-8c84-3e3a7a6f4bba] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:31.830972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:31.832107 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:31.832885 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:31.833312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:31.847224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] HTTP exception thrown: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Request method failure captured: May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba HTTP/1.1 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjthl0myhmqPmiGTeiPTvzXbtMpWDMxPdif8Pf5emAeJavRV-vxeFPtuJ0wYYBgnk2IXcYU37-CczZddQz4UP4i2ovW97CgIrBXGek_HbqVGjAexAtodIYVem0ByrQiQh3t1taOziVyOLArxwcchwMFbA-aHR7bBwc5EgoNRQQ5gLY0 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: cedf35fd585743f2893b58a2fda9edff May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-VolumesActionsTest-1411779430 May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:43:31.848949 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/cedf35fd585743f2893b58a2fda9edff"}]}] May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-VolumesActionsTest-1411779430 May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: cedf35fd585743f2893b58a2fda9edff May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-VolumesActionsTest-1411779430 May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-VolumesActionsTest-1411779430-project May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 912741d6d24b42d5bf98cf0d2cc8e43d May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-VolumesActionsTest-1411779430-project May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. May 19 12:43:31.850768 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '4330061e-acfa-4968-8c84-3e3a7a6f4bba'} May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:31.852231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. May 19 12:43:31.853988 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:31.855330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Returning 404 to user: Instance 4330061e-acfa-4968-8c84-3e3a7a6f4bba could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:31.855330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-db899421-c822-4b9f-b100-b189c468769e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba" status: 404 len: 111 microversion: 2.1 time: 0.030793 May 19 12:43:31.855330 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1664/3216] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:31 2022] GET /compute/v2.1/servers/4330061e-acfa-4968-8c84-3e3a7a6f4bba => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:43:31.901793 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1133825360", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "networks": [{"uuid": "6bbeebae-8969-43b5-b49f-81ba4270e2d1"}]}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:31.954608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.network.neutron [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] validate_networks() for [('6bbeebae-8969-43b5-b49f-81ba4270e2d1', None, None, None, None, None)] {{(pid=117532) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:43:32.336909 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:32.337903 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:32.344180 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.344967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.345973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.346614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.432286 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lazy-loading 'services' on Instance uuid 4be06310-2d56-49f9-8a1a-e461b115a32b {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:32.486789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Flavor limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:43:32.487507 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Image limits 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:43:32.488181 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Flavor pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:43:32.489113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Image pref 0:0:0 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:43:32.489814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.virt.hardware [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117532) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:43:32.512764 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.513555 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.514850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.515569 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.521217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-11552ce1-e417-4ea5-a338-f8a2396170a4 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments/5239592d-4a9e-4fe4-bbd3-859e6a01f961" status: 202 len: 0 microversion: 2.1 time: 0.188735 May 19 12:43:32.522767 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1665/3217] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:43:32 2022] DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments/5239592d-4a9e-4fe4-bbd3-859e6a01f961 => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:43:32.530138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}, {"name": "network-changed", "server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:32.541117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Getting quotas for project cedf35fd585743f2893b58a2fda9edff. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:43:32.543599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.544131 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.546477 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Getting quotas for user 912741d6d24b42d5bf98cf0d2cc8e43d and project cedf35fd585743f2893b58a2fda9edff. Resources: {'instances', 'cores', 'ram'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:43:32.561497 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] Creating event network-changed:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:43:32.561730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Going to run 1 instances... {{(pid=117532) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:43:32.562430 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] Creating event network-changed:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:43:32.567816 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-24d240e6-9309-4e35-b3e2-bd52143dd1b8 req-523a3dda-83c2-44d6-a4d2-8586017325ef service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.042259 May 19 12:43:32.568432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1666/3218] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:32 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:32.573497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117532) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:43:32.584358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:32.585431 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:32.591874 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.593200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.594166 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.594789 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.641337 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e76b3e3-d57a-4eb9-95b0-72a56f1a4dfb tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.744961 May 19 12:43:32.647998 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1553/3219] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:43:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 754 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:43:32.653877 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:32.655987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:32.660052 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-33603908-f2d8-4359-8e4e-027d78ff205f tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085971 May 19 12:43:32.660167 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1667/3220] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:43:32 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:32.665424 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.666121 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.667444 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:32.668283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:32.672050 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac6c0614-0e32-402b-9844-0e51f3f4864f tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:32.672687 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac6c0614-0e32-402b-9844-0e51f3f4864f tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:32.752474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-ac6c0614-0e32-402b-9844-0e51f3f4864f tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:32.760666 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac6c0614-0e32-402b-9844-0e51f3f4864f tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1214 microversion: 2.1 time: 0.099527 May 19 12:43:32.761609 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1668/3221] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:32 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1214 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:32.839251 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:32.846893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-24609a97-2c11-49bb-a54f-ac67bcd9a3cd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.197477 May 19 12:43:32.847699 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1554/3222] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:32 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:33.671969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:33.673007 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:33.681773 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.682293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.683110 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.683536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.730011 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-af1683ef-7f56-4e2e-af1e-82dc7725a511 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.062568 May 19 12:43:33.735321 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1669/3223] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:43:33 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments => generated 25 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:43:33.742004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:33.743330 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:33.750087 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.750534 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.751295 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.751719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.778778 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:33.779454 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:33.786075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.786522 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.787293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.787755 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.837770 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] HTTP exception thrown: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b is not attached to volume 5239592d-4a9e-4fe4-bbd3-859e6a01f961 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Request method failure captured: May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments/5239592d-4a9e-4fe4-bbd3-859e6a01f961 HTTP/1.1 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjrir0fYW1Sc1ZjPY9P2WdTPlTQFKY5i7bYc-yj3GO-b6LtQ2mHkGh1LT6dz9tX6NR-jj0UcO3XhGiwTYn8zETAqWF5Ek2XeR8Pq1mn_KK2JUQ8-A33Ruhu-2gKxTzojQJcanJWByGEtT7AmYlH7YwGygiBMSAssyPrzVPNd9EwH9gU May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: a76f72af8d47404980293788d97edb2e May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:43:33.840413 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a76f72af8d47404980293788d97edb2e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a76f72af8d47404980293788d97edb2e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a76f72af8d47404980293788d97edb2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a76f72af8d47404980293788d97edb2e"}]}] May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: a76f72af8d47404980293788d97edb2e May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-TestMinimumBasicScenario-428002016-project May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: f725498599d84b3782c0e0d46131463a May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-TestMinimumBasicScenario-428002016-project May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:43:33.842113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b is not attached to volume 5239592d-4a9e-4fe4-bbd3-859e6a01f961 May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'server_id': '4be06310-2d56-49f9-8a1a-e461b115a32b', 'id': '5239592d-4a9e-4fe4-bbd3-859e6a01f961'} May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 5239592d-4a9e-4fe4-bbd3-859e6a01f961. May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:33.842990 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b is not attached to volume 5239592d-4a9e-4fe4-bbd3-859e6a01f961 May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b is not attached to volume 5239592d-4a9e-4fe4-bbd3-859e6a01f961 May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:33.845153 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Returning 404 to user: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b is not attached to volume 5239592d-4a9e-4fe4-bbd3-859e6a01f961 {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:33.848410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-15e54323-a6c0-4a88-8b96-148e6640fbe1 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments/5239592d-4a9e-4fe4-bbd3-859e6a01f961" status: 404 len: 154 microversion: 2.1 time: 0.105308 May 19 12:43:33.848410 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1555/3224] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:43:33 2022] DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b/os-volume_attachments/5239592d-4a9e-4fe4-bbd3-859e6a01f961 => generated 154 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 19 12:43:33.863329 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:33.864163 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:33.870775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.871213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.871969 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:33.872387 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:33.912306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:33.922603 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8b810963-70a7-41eb-a8a3-a09d79cf51a9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1274 microversion: 2.1 time: 0.148343 May 19 12:43:33.936807 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1670/3225] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:33 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1274 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:34.139247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:34.147510 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-28e38e57-48f1-4685-9ca6-437d1d4e5a55 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.287970 May 19 12:43:34.148296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1556/3226] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:33 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:34.943028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:34.943695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:34.952853 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:34.953298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:34.954060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:34.954489 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.021404 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:35.021884 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:35.030675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.031117 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.032113 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.032772 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.058892 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:43:35.104527 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:35.112305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-57c298fd-7d5a-450d-af3d-111a7cc74b0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.173202 May 19 12:43:35.113150 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1671/3227] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:34 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:35.147929 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-396fc1c6-dd72-4b9c-b808-70b52ad4b356 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 204 len: 0 microversion: 2.1 time: 0.130000 May 19 12:43:35.152396 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1557/3228] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:43:35 2022] DELETE /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 19 12:43:35.157561 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:35.158172 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:35.164352 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.164814 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:35.164954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.165570 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.165693 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:35.165995 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.172765 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.173812 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.174578 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.175005 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.293956 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:35.301960 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4be4b20d-6978-428d-afe7-d00ea4eebbf0 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1554 microversion: 2.1 time: 0.148372 May 19 12:43:35.302523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1672/3229] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:35 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1554 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:35.354462 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-c8395f82-8c9d-45fe-b8b0-6f98ae3935a9 req-950f9fbc-b012-4d6d-8f92-325387a81828 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "317f43b8-11c3-4e5f-b182-0688e3a701fb", "tag": "594e80bc-7f3f-4565-8091-abeef6e769fd"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:35.362724 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-c8395f82-8c9d-45fe-b8b0-6f98ae3935a9 req-950f9fbc-b012-4d6d-8f92-325387a81828 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:35.363154 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-c8395f82-8c9d-45fe-b8b0-6f98ae3935a9 req-950f9fbc-b012-4d6d-8f92-325387a81828 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:35.374696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-c8395f82-8c9d-45fe-b8b0-6f98ae3935a9 req-950f9fbc-b012-4d6d-8f92-325387a81828 service nova] Creating event network-changed:594e80bc-7f3f-4565-8091-abeef6e769fd for instance 317f43b8-11c3-4e5f-b182-0688e3a701fb on hv-cinder-80403 May 19 12:43:35.379480 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-c8395f82-8c9d-45fe-b8b0-6f98ae3935a9 req-950f9fbc-b012-4d6d-8f92-325387a81828 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028770 May 19 12:43:35.380080 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1673/3230] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:35.393099 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:35.402429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-4de4cacf-8a6b-4e89-a31a-93576d2cd7b4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.241150 May 19 12:43:35.402429 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1558/3231] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:35 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:36.131400 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:36.132059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:36.138729 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.139206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.139936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.140360 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.272384 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:36.280645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-6728072b-c851-40f1-87cc-9511bdd1983c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.153950 May 19 12:43:36.281224 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1674/3232] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:36 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:36.318873 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:36.319493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:36.326571 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.327127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.327750 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.328170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.418826 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:36.419423 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:36.426182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.426698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.427675 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:36.428139 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:36.459315 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:36.467333 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1d74ccd2-1dfc-488b-80aa-7706669bdb32 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1554 microversion: 2.1 time: 0.152259 May 19 12:43:36.467895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1559/3233] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:36 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1554 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:36.560217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:36.571879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-582e607b-1f53-4ce6-a078-0bf02ae0c8e1 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.156767 May 19 12:43:36.572557 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1675/3234] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:36 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:37.297665 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:37.298649 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:37.305426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.305951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.306991 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.307401 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.311675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:37.476004 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-b729b96b-c832-4e5f-ac95-4e6ae178721e req-aa7649b4-b477-4f23-9510-2d5b9e1bb8cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4be06310-2d56-49f9-8a1a-e461b115a32b", "name": "network-vif-deleted", "tag": "fedd35b6-9b61-414c-b8f9-173f39f64c6e"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:37.484717 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b729b96b-c832-4e5f-ac95-4e6ae178721e req-aa7649b4-b477-4f23-9510-2d5b9e1bb8cc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.485429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b729b96b-c832-4e5f-ac95-4e6ae178721e req-aa7649b4-b477-4f23-9510-2d5b9e1bb8cc service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.497254 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-b729b96b-c832-4e5f-ac95-4e6ae178721e req-aa7649b4-b477-4f23-9510-2d5b9e1bb8cc service nova] Creating event network-vif-deleted:fedd35b6-9b61-414c-b8f9-173f39f64c6e for instance 4be06310-2d56-49f9-8a1a-e461b115a32b on hv-cinder-80403 May 19 12:43:37.502460 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-b729b96b-c832-4e5f-ac95-4e6ae178721e req-aa7649b4-b477-4f23-9510-2d5b9e1bb8cc service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030276 May 19 12:43:37.503316 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1676/3235] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:43:37.508697 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:37.510210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:37.517684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.518374 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.519133 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.519723 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.535441 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:37.547754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8e2a30e7-ae04-4434-9726-ed93da31f2c9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.254279 May 19 12:43:37.558303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1560/3236] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:37 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:37.588271 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:37.592233 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:37.593294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:37.597950 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-2343109f-3b98-4e15-a693-390e8c103761 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1344 microversion: 2.1 time: 0.093232 May 19 12:43:37.598855 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1677/3237] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:37 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1344 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:37.605543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.606255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.607286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.607931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.718170 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-b44723b3-2fb1-47cf-a853-3aa314e8aaec req-fb2c1ee4-fd91-4d2d-8722-67ff586bc336 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7b74f125-b188-4751-baed-8d0bb14ded3b", "name": "network-vif-plugged", "status": "completed", "tag": "877681eb-a6ae-4d76-8be0-e607037850b1"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:37.727636 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b44723b3-2fb1-47cf-a853-3aa314e8aaec req-fb2c1ee4-fd91-4d2d-8722-67ff586bc336 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:37.728105 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-b44723b3-2fb1-47cf-a853-3aa314e8aaec req-fb2c1ee4-fd91-4d2d-8722-67ff586bc336 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:37.743830 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-b44723b3-2fb1-47cf-a853-3aa314e8aaec req-fb2c1ee4-fd91-4d2d-8722-67ff586bc336 service nova] Creating event network-vif-plugged:877681eb-a6ae-4d76-8be0-e607037850b1 for instance 7b74f125-b188-4751-baed-8d0bb14ded3b on hv-cinder-80403 May 19 12:43:37.752192 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-b44723b3-2fb1-47cf-a853-3aa314e8aaec req-fb2c1ee4-fd91-4d2d-8722-67ff586bc336 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037669 May 19 12:43:37.753314 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1678/3238] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:43:38.006737 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:38.014196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-de82807c-5764-4031-8c42-f0a86f87be6a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.428941 May 19 12:43:38.014780 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1561/3239] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:37 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:43:38.570341 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:38.571033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:38.578936 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:38.579439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:38.580226 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:38.580664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:38.615079 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:38.616071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:38.623102 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:38.623543 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:38.624296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:38.624757 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:38.657320 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:38.657759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:38.658207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lazy-loading 'fault' on Instance uuid 4be06310-2d56-49f9-8a1a-e461b115a32b {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:38.704159 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f725498599d84b3782c0e0d46131463a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a76f72af8d47404980293788d97edb2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:38.712872 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3c4c8ae0-60bd-46f8-9a0b-8433e0bda3ee tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 200 len: 1349 microversion: 2.1 time: 0.101480 May 19 12:43:38.713751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1562/3240] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:38 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:38.722312 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:38.732058 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5ef60709-f19e-45c3-a888-82d0cbcccef2 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.169302 May 19 12:43:38.733599 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1679/3241] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:38 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:39.035041 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:39.035949 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:39.047040 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.047769 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.049132 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.049836 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.186459 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:39.198548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-f68f7656-11be-44b2-ba11-e9d071a22823 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1367 microversion: 2.1 time: 0.169785 May 19 12:43:39.199349 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1563/3242] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:39 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1367 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:39.734425 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:39.735381 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] [instance: 4be06310-2d56-49f9-8a1a-e461b115a32b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:39.745109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.745565 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.746317 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.746749 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.751426 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:39.752071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:39.759460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.760150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.761057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] HTTP exception thrown: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. May 19 12:43:39.761267 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:39.761579 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Request method failure captured: May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b HTTP/1.1 May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjrir0fYW1Sc1ZjPY9P2WdTPlTQFKY5i7bYc-yj3GO-b6LtQ2mHkGh1LT6dz9tX6NR-jj0UcO3XhGiwTYn8zETAqWF5Ek2XeR8Pq1mn_KK2JUQ8-A33Ruhu-2gKxTzojQJcanJWByGEtT7AmYlH7YwGygiBMSAssyPrzVPNd9EwH9gU May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: a76f72af8d47404980293788d97edb2e May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:43:39.762622 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a76f72af8d47404980293788d97edb2e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a76f72af8d47404980293788d97edb2e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a76f72af8d47404980293788d97edb2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a76f72af8d47404980293788d97edb2e"}]}] May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: a76f72af8d47404980293788d97edb2e May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-TestMinimumBasicScenario-428002016 May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-TestMinimumBasicScenario-428002016-project May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: f725498599d84b3782c0e0d46131463a May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-TestMinimumBasicScenario-428002016-project May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. May 19 12:43:39.774852 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '4be06310-2d56-49f9-8a1a-e461b115a32b'} May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:39.776061 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. May 19 12:43:39.777932 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:43:39.779601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Returning 404 to user: Instance 4be06310-2d56-49f9-8a1a-e461b115a32b could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:39.779601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-370261eb-852f-4375-9237-9cf0ce1d4049 tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b" status: 404 len: 111 microversion: 2.1 time: 0.036240 May 19 12:43:39.779601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1680/3243] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:39 2022] GET /compute/v2.1/servers/4be06310-2d56-49f9-8a1a-e461b115a32b => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:43:39.779601 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ec6133c0-d241-4566-803b-ee7e3fac7a9b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] Calling method '.version_select at 0x7fe211347280>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:39.795764 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ec6133c0-d241-4566-803b-ee7e3fac7a9b tempest-TestMinimumBasicScenario-428002016 tempest-TestMinimumBasicScenario-428002016-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1665217945" status: 202 len: 0 microversion: 2.1 time: 0.026284 May 19 12:43:39.800396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1681/3244] 10.222.0.35 () {58 vars in 1271 bytes} [Thu May 19 12:43:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1665217945 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:43:39.924581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:39.932294 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-40c06daf-ad87-4134-8828-e6a3101eac85 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.184872 May 19 12:43:39.932942 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1564/3245] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:39 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:40.215642 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:40.216313 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:40.224349 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.224883 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:40.226016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.226442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:40.362223 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:40.370298 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d6d249e7-edf0-422f-8949-6ccffcc6a6a5 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1558 microversion: 2.1 time: 0.158628 May 19 12:43:40.370861 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1682/3246] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:40 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:40.386638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:40.387358 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:40.393548 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.394052 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:40.394864 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.395409 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:40.473255 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-febb0389-06d1-4f58-aa1c-fdbc842a952f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.090371 May 19 12:43:40.473853 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1565/3247] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:43:40 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:43:40.950112 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:40.950842 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:40.957684 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.958277 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:40.959145 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:40.959696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:41.174719 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:41.182536 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d248cec7-39f2-4aa4-b5d1-2457e53e71ca tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.236602 May 19 12:43:41.183393 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1683/3248] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:40 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:41.410790 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-80a1a1e0-16f8-43ea-a558-fbbe00f9e55d req-38b02a71-151f-4af8-bdd7-83e0ee6243d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7b74f125-b188-4751-baed-8d0bb14ded3b", "tag": "877681eb-a6ae-4d76-8be0-e607037850b1"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:41.423066 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-80a1a1e0-16f8-43ea-a558-fbbe00f9e55d req-38b02a71-151f-4af8-bdd7-83e0ee6243d7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:41.423554 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-80a1a1e0-16f8-43ea-a558-fbbe00f9e55d req-38b02a71-151f-4af8-bdd7-83e0ee6243d7 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:41.435920 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-80a1a1e0-16f8-43ea-a558-fbbe00f9e55d req-38b02a71-151f-4af8-bdd7-83e0ee6243d7 service nova] Creating event network-changed:877681eb-a6ae-4d76-8be0-e607037850b1 for instance 7b74f125-b188-4751-baed-8d0bb14ded3b on hv-cinder-80403 May 19 12:43:41.442147 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-80a1a1e0-16f8-43ea-a558-fbbe00f9e55d req-38b02a71-151f-4af8-bdd7-83e0ee6243d7 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036577 May 19 12:43:41.443456 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1566/3249] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:42.199549 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:42.200297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:42.206993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:42.207440 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:42.208429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:42.209068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:42.233320 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:42.752671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:42.756067 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:42.763483 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-8ab300fc-c55b-4f17-b3d8-8daaeeb1c244 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.567956 May 19 12:43:42.766466 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1684/3250] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:42 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:43.780212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:43.782632 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:43.785557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:43.786581 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=22 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:43.793680 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:43.794110 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:43.800564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:43.800564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:43.847779 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-8670255f-18be-4d97-a282-1a5596feaf8e req-28e62cd8-e9dc-4673-bcba-0b5f36cce6f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "317f43b8-11c3-4e5f-b182-0688e3a701fb", "name": "network-vif-plugged", "status": "completed", "tag": "594e80bc-7f3f-4565-8091-abeef6e769fd"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:43.870382 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:43.872441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-8670255f-18be-4d97-a282-1a5596feaf8e req-28e62cd8-e9dc-4673-bcba-0b5f36cce6f4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:43.872829 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-8670255f-18be-4d97-a282-1a5596feaf8e req-28e62cd8-e9dc-4673-bcba-0b5f36cce6f4 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:43.891993 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-8670255f-18be-4d97-a282-1a5596feaf8e req-28e62cd8-e9dc-4673-bcba-0b5f36cce6f4 service nova] Creating event network-vif-plugged:594e80bc-7f3f-4565-8091-abeef6e769fd for instance 317f43b8-11c3-4e5f-b182-0688e3a701fb on hv-cinder-80403 May 19 12:43:43.904596 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-8670255f-18be-4d97-a282-1a5596feaf8e req-28e62cd8-e9dc-4673-bcba-0b5f36cce6f4 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060488 May 19 12:43:43.905591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1685/3251] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 19 12:43:43.981131 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:43.997472 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-80562441-f1cf-41c3-9e05-34e0cb3f983a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.221138 May 19 12:43:44.006713 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1567/3252] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:43 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:45.015414 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:45.016182 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:45.022780 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:45.023652 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:45.024305 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:45.024888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:45.176200 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:45.184663 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4f7759d4-49a0-46aa-bd58-a103174026f8 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1314 microversion: 2.1 time: 0.173891 May 19 12:43:45.185379 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1686/3253] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:45 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:46.201719 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:46.203207 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:46.209644 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:46.210046 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:46.210787 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:46.211418 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:46.499598 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:46.507965 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d591136d-2614-449a-b358-ed00cd8b52f7 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1499 microversion: 2.1 time: 0.310043 May 19 12:43:46.508553 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1568/3254] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:46 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1499 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:47.043646 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:47.044155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:47.050736 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:47.051179 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:47.051930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:47.052359 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:47.077113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:43:47.174631 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5690ac35-fe4a-4bc4-b2c0-3eacdb972df9 tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "DELETE /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 204 len: 0 microversion: 2.1 time: 0.134606 May 19 12:43:47.175537 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1687/3255] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:43:47 2022] DELETE /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:43:47.184931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:47.185894 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:47.194112 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:47.195128 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:47.195913 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:47.196395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:47.410059 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:47.418293 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-d51479cb-b948-4d08-a3e6-e3e5e4c8522e tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1505 microversion: 2.1 time: 0.238331 May 19 12:43:47.423522 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1569/3256] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:47 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1505 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:48.437109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:48.438094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:48.446185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:48.446739 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:48.447491 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:48.447929 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:48.594469 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:48.602888 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-158e972a-0b70-4589-8e5e-cfb8f1aabc0a tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1505 microversion: 2.1 time: 0.170961 May 19 12:43:48.603502 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1688/3257] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:48 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1505 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:49.335490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-f0f69e24-f289-4ba9-8977-2774abff5f7b req-b3aa2155-e222-4d8c-9783-53bd7394ad44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "317f43b8-11c3-4e5f-b182-0688e3a701fb", "name": "network-vif-deleted", "tag": "594e80bc-7f3f-4565-8091-abeef6e769fd"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:43:49.343497 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-f0f69e24-f289-4ba9-8977-2774abff5f7b req-b3aa2155-e222-4d8c-9783-53bd7394ad44 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:49.343936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-f0f69e24-f289-4ba9-8977-2774abff5f7b req-b3aa2155-e222-4d8c-9783-53bd7394ad44 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:49.357739 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-f0f69e24-f289-4ba9-8977-2774abff5f7b req-b3aa2155-e222-4d8c-9783-53bd7394ad44 service nova] Creating event network-vif-deleted:594e80bc-7f3f-4565-8091-abeef6e769fd for instance 317f43b8-11c3-4e5f-b182-0688e3a701fb on hv-cinder-80403 May 19 12:43:49.364882 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-f0f69e24-f289-4ba9-8977-2774abff5f7b req-b3aa2155-e222-4d8c-9783-53bd7394ad44 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.218002 May 19 12:43:49.365921 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1570/3258] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:43:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:43:49.619872 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:49.620741 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:49.627783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:49.628217 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:49.629012 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:49.629446 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:49.662802 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:49.663248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:49.663750 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lazy-loading 'fault' on Instance uuid 317f43b8-11c3-4e5f-b182-0688e3a701fb {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:43:49.706721 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '912741d6d24b42d5bf98cf0d2cc8e43d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cedf35fd585743f2893b58a2fda9edff', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:43:49.713634 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-04cbd156-9424-4d60-babd-53d200a8608c tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 200 len: 1304 microversion: 2.1 time: 0.098058 May 19 12:43:49.714194 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1689/3259] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:49 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 1304 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:43:50.733177 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:43:50.733759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] [instance: 317f43b8-11c3-4e5f-b182-0688e3a701fb] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:43:50.740520 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:50.740988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:50.741878 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:43:50.742299 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:43:50.756123 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] HTTP exception thrown: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Request method failure captured: May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb HTTP/1.1 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjthl0myhmqPmiGTeiPTvzXbtMpWDMxPdif8Pf5emAeJavRV-vxeFPtuJ0wYYBgnk2IXcYU37-CczZddQz4UP4i2ovW97CgIrBXGek_HbqVGjAexAtodIYVem0ByrQiQh3t1taOziVyOLArxwcchwMFbA-aHR7bBwc5EgoNRQQ5gLY0 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: cedf35fd585743f2893b58a2fda9edff May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-VolumesActionsTest-1411779430 May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:43:50.757847 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_cedf35fd585743f2893b58a2fda9edff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/cedf35fd585743f2893b58a2fda9edff"}]}] May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-VolumesActionsTest-1411779430 May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: cedf35fd585743f2893b58a2fda9edff May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-VolumesActionsTest-1411779430 May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-VolumesActionsTest-1411779430-project May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 912741d6d24b42d5bf98cf0d2cc8e43d May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-VolumesActionsTest-1411779430-project May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. May 19 12:43:50.759637 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '317f43b8-11c3-4e5f-b182-0688e3a701fb'} May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:43:50.760594 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. May 19 12:43:50.762218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:43:50.763675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] Returning 404 to user: Instance 317f43b8-11c3-4e5f-b182-0688e3a701fb could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:43:50.763675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-01cdda7e-727b-4f3e-aa58-3d54d882cbde tempest-VolumesActionsTest-1411779430 tempest-VolumesActionsTest-1411779430-project] 10.222.0.35 "GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb" status: 404 len: 111 microversion: 2.1 time: 0.031008 May 19 12:43:50.763675 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1571/3260] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:43:50 2022] GET /compute/v2.1/servers/317f43b8-11c3-4e5f-b182-0688e3a701fb => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:44:25.090236 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:25.091233 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:25.094761 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:25.095796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:25.104084 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:25.104523 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:25.105293 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:25.105695 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:25.131419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-27515196-4114-4b65-848e-cc617843b063 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/ips" status: 200 len: 151 microversion: 2.1 time: 0.047544 May 19 12:44:25.131907 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1690/3261] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:44:25 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/ips => generated 151 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:27.516931 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "254fd44f-890d-42b1-a74a-21f766ad48c5"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:27.518286 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:27.521667 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:27.522434 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=58 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:27.528055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:27.528499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:27.529490 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:27.529925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:27.804421 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-60245c06-00ab-42de-bacc-81c3fb91ef1e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.291201 May 19 12:44:27.805244 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1572/3262] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:44:27 2022] POST /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments => generated 194 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:35.363136 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "254fd44f-890d-42b1-a74a-21f766ad48c5"}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:35.364815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:35.367935 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:35.368562 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=4 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:35.374268 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.374696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.375442 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.375857 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.456674 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: POST /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments HTTP/1.1 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 74 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:44:35.458434 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {"volumeAttachment": {"volumeId": "254fd44f-890d-42b1-a74a-21f766ad48c5"}} May 19 12:44:35.459534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'server_id': '7b74f125-b188-4751-baed-8d0bb14ded3b', 'body': {'volumeAttachment': {'volumeId': '254fd44f-890d-42b1-a74a-21f766ad48c5'}}} May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:44:35.460221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:35.461155 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 400 to user: Invalid volume: volume 254fd44f-890d-42b1-a74a-21f766ad48c5 is already attached to instances: 7b74f125-b188-4751-baed-8d0bb14ded3b {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-ac44ca67-46dc-48e9-9763-7d8a8cd050c0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.100788 May 19 12:44:35.461997 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1691/3263] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:44:35 2022] POST /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 19 12:44:35.523834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:35.524529 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:35.534351 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.534981 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.536154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.536834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.626755 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.objects.instance [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lazy-loading 'services' on Instance uuid 7b74f125-b188-4751-baed-8d0bb14ded3b {{(pid=117532) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:44:35.704616 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-8dbff25a-2c0e-4a36-a21e-21debba357e4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments/254fd44f-890d-42b1-a74a-21f766ad48c5" status: 202 len: 0 microversion: 2.1 time: 0.185542 May 19 12:44:35.705714 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1573/3264] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:44:35 2022] DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments/254fd44f-890d-42b1-a74a-21f766ad48c5 => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:44:35.714185 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:35.715198 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:35.721716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.722140 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.722930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:35.723326 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:35.755461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-1dcf7689-e97b-4a95-9b20-0b4e5c1a50dd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045226 May 19 12:44:35.756023 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1692/3265] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:44:35 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:36.769082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:36.770283 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:36.777754 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:36.778189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:36.778936 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:36.779360 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:36.808280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6ab8ef82-c86f-4500-bdbf-768e7fa82280 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044061 May 19 12:44:36.808850 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1574/3266] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:44:36 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:44:36.865619 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:36.866113 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:36.872967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:36.873407 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:36.874146 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:36.874566 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:36.902206 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Going to try to terminate instance {{(pid=117533) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:44:36.981395 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-c249074b-6139-4a13-a55f-9bd7e235dcd6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 204 len: 0 microversion: 2.1 time: 0.119613 May 19 12:44:36.981965 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1693/3267] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:44:36 2022] DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:44:36.992775 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:36.993377 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:36.999493 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:36.999932 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:37.000687 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:37.001127 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:37.139335 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:37.152023 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ab5ee501-d7c3-47bc-bdeb-15a32502fc2e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1683 microversion: 2.1 time: 0.164952 May 19 12:44:37.152867 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1575/3268] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:36 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1683 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:38.179410 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:38.180401 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:38.186967 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:38.187411 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:38.188160 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:38.188585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:38.324702 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:38.338973 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d8baa4ae-ec96-4a35-99cc-88f518a8da7b tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1683 microversion: 2.1 time: 0.166891 May 19 12:44:38.339569 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1694/3269] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:38 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1683 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:39.357213 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:39.358019 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:39.367563 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:39.368011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:39.368802 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:39.369479 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:39.466676 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:39.474983 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-74f37c8a-aeec-4ccf-afb6-0d0728f25b36 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 200 len: 1615 microversion: 2.1 time: 0.122152 May 19 12:44:39.475834 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1576/3270] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:39 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 1615 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:39.691408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-75abda84-07ee-4c7f-b888-cbc40b16e104 req-ed865f40-620e-4b42-ac1c-001167e6c7b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7b74f125-b188-4751-baed-8d0bb14ded3b", "name": "network-vif-deleted", "tag": "877681eb-a6ae-4d76-8be0-e607037850b1"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:39.702608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-75abda84-07ee-4c7f-b888-cbc40b16e104 req-ed865f40-620e-4b42-ac1c-001167e6c7b9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:39.703036 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-75abda84-07ee-4c7f-b888-cbc40b16e104 req-ed865f40-620e-4b42-ac1c-001167e6c7b9 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:39.715057 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-75abda84-07ee-4c7f-b888-cbc40b16e104 req-ed865f40-620e-4b42-ac1c-001167e6c7b9 service nova] Creating event network-vif-deleted:877681eb-a6ae-4d76-8be0-e607037850b1 for instance 7b74f125-b188-4751-baed-8d0bb14ded3b on hv-cinder-80403 May 19 12:44:39.720626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-75abda84-07ee-4c7f-b888-cbc40b16e104 req-ed865f40-620e-4b42-ac1c-001167e6c7b9 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033151 May 19 12:44:39.724396 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1695/3271] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:44:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:40.493575 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:40.494212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:40.500848 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:40.501523 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:40.502280 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:40.502705 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:40.519385 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b HTTP/1.1 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:44:40.521006 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:44:40.522082 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '7b74f125-b188-4751-baed-8d0bb14ded3b'} May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:44:40.522638 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:44:40.523736 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:44:40.524620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:44:40.524620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-660bc844-d1c4-496b-81cf-e5bdb761eee9 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 404 len: 111 microversion: 2.1 time: 0.033077 May 19 12:44:40.524620 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1577/3272] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:40 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:44:40.531996 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-46148fab-5f2c-4614-8179-e1fc02be0df4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '.version_select at 0x7fe20c07adc0>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:40.551439 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-46148fab-5f2c-4614-8179-e1fc02be0df4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-653431057" status: 202 len: 0 microversion: 2.1 time: 0.023591 May 19 12:44:40.552118 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1696/3273] 10.222.0.35 () {58 vars in 1217 bytes} [Thu May 19 12:44:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-653431057 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:44:41.421303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-0270a2f3-7062-4177-860c-3ccbf90c4dde tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: .version_select at 0x7fe20c586040>, body: {"keypair": {"name": "tempest-keypair-1811807841", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA5v08mvP5NYPPrj3Hwi9BmLhMneLSu9g6zZeIMJSiEvvZ9/5jE8WbuiFzbhBVyP/Bj9viFub1t4ZbjVvHJ5TIjjH9BWKe+/1J0LXZT7ELGNiUxsxh97Q1SzzxGpV8RP7Q=="}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:41.435260 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-0270a2f3-7062-4177-860c-3ccbf90c4dde tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:44:41.439724 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.quota [None req-0270a2f3-7062-4177-860c-3ccbf90c4dde tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'key_pairs'} {{(pid=117532) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:44:41.464266 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-0270a2f3-7062-4177-860c-3ccbf90c4dde tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.046919 May 19 12:44:41.464908 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1578/3274] 10.222.0.35 () {60 vars in 1158 bytes} [Thu May 19 12:44:41 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:42.798281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1717172443", "imageRef": "21a7b388-b833-4b08-994b-7af14484df6f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f504b23c-8d0f-4020-83c4-474e12095c43"}], "security_groups": [{"name": "tempest-securitygroup--435586935"}], "key_name": "tempest-keypair-1811807841", "user_data": "}} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:42.905119 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.network.neutron [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] validate_networks() for [('f504b23c-8d0f-4020-83c4-474e12095c43', None, None, None, None, None)] {{(pid=117533) validate_networks /opt/stack/nova/nova/network/neutron.py:2608}} May 19 12:44:43.378033 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 19 12:44:43.378441 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image limits 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 19 12:44:43.378902 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Flavor pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 19 12:44:43.379322 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Image pref 0:0:0 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 19 12:44:43.379700 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.virt.hardware [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117533) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 19 12:44:43.389543 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:43.389972 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:43.390745 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:43.391163 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:43.408796 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 19 12:44:43.413427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.quota [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Getting quotas for user 1aad8b1af5994d7b9de3a8fb0be556cf and project 0f0a91a3d6a14259b2d03881fdab4f9f. Resources: {'instances', 'cores', 'ram'} {{(pid=117533) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 19 12:44:43.426062 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Going to run 1 instances... {{(pid=117533) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 19 12:44:43.428408 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='21a7b388-b833-4b08-994b-7af14484df6f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117533) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 19 12:44:43.476264 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [23d5f452-058d-4548-aac1-ac1e6c7941c5] AMQP server on 10.222.0.35:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 19 12:44:44.490077 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [23d5f452-058d-4548-aac1-ac1e6c7941c5] Reconnected to AMQP server on 10.222.0.35:5672 via [amqp] client with port 52274. May 19 12:44:44.496100 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4149db26-4355-43f9-926c-8a46f70c523f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.702885 May 19 12:44:44.496954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1697/3275] 10.222.0.35 () {60 vars in 1146 bytes} [Thu May 19 12:44:42 2022] POST /compute/v2.1/servers => generated 401 bytes in 1704 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 19 12:44:44.507653 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1e45b323-eb75-4196-b998-b278677faf92 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:44.509540 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1e45b323-eb75-4196-b998-b278677faf92 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:44.577196 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1e45b323-eb75-4196-b998-b278677faf92 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:44.583934 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1e45b323-eb75-4196-b998-b278677faf92 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1242 microversion: 2.1 time: 0.080888 May 19 12:44:44.584494 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1579/3276] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:44 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1242 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:45.602373 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:45.603069 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:45.609858 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:45.610378 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:45.611138 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:45.611626 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:45.681614 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:45.690947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-d89dec3b-7f30-4620-9bfd-2930fff1fc06 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1302 microversion: 2.1 time: 0.092984 May 19 12:44:45.692060 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1698/3277] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:45 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1302 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:46.708222 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:46.708895 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:46.716323 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:46.716803 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:46.717562 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:46.718007 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:46.924937 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:46.932946 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1964f0dc-66e7-47b7-8ca6-828460f75f23 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.229083 May 19 12:44:46.933486 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1580/3278] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:46 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 19 12:44:47.222814 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [req-e66aefd7-75df-4244-a468-63607f5a2c33 req-fc70a8a7-f632-4473-9aee-f5a71c61594d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01069733-00c3-4faa-a755-f7e224ade18a", "tag": "0ed79937-38b1-45ad-8dee-edbb5ccb7ea4"}]} {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:47.233856 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-e66aefd7-75df-4244-a468-63607f5a2c33 req-fc70a8a7-f632-4473-9aee-f5a71c61594d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:47.234348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [req-e66aefd7-75df-4244-a468-63607f5a2c33 req-fc70a8a7-f632-4473-9aee-f5a71c61594d service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:47.260501 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.compute.server_external_events [req-e66aefd7-75df-4244-a468-63607f5a2c33 req-fc70a8a7-f632-4473-9aee-f5a71c61594d service nova] Creating event network-changed:0ed79937-38b1-45ad-8dee-edbb5ccb7ea4 for instance 01069733-00c3-4faa-a755-f7e224ade18a on hv-cinder-80403 May 19 12:44:47.265432 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [req-e66aefd7-75df-4244-a468-63607f5a2c33 req-fc70a8a7-f632-4473-9aee-f5a71c61594d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046278 May 19 12:44:47.265974 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1699/3279] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:44:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:47.954570 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:47.955447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:47.966492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:47.967227 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:47.968492 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:47.969239 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:48.107460 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:48.119114 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-df351d8e-614c-477b-822a-27c0a8cf7719 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.171184 May 19 12:44:48.119890 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1581/3280] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:47 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:49.146831 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:49.147969 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:49.154870 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:49.155311 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:49.156099 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:49.156525 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:49.287250 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:49.298540 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7508edc4-9033-449c-91c5-d6130e1eca7d tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.159446 May 19 12:44:49.299318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1700/3281] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:49 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:50.321359 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:50.322062 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:50.330096 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:50.330612 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:50.331485 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:50.332004 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:50.476499 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:50.484285 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-e1fe0894-9919-4173-b587-ed63fe16e4fd tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.169117 May 19 12:44:50.484902 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1582/3282] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:50 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:51.505947 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:51.507851 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:51.518758 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:51.519481 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:51.520725 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:51.521445 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:51.669911 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:51.681302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aba92279-bf94-47d9-bbcb-a9b65c1bcca0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.181739 May 19 12:44:51.682237 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1701/3283] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:51 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:52.707322 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:52.708271 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:52.719288 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:52.720010 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:52.721303 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:52.721999 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:52.868250 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:52.882254 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a7131850-4d16-4ab1-b792-4deede7c6e3e tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.181366 May 19 12:44:52.882826 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1583/3284] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:52 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:53.903297 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:53.904270 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:53.908331 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=2 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:53.909533 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell0:SELECT=6 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:53.920306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:53.920306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:53.920306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:53.920306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:54.055443 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:54.063342 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-f0c1c225-a68c-45e9-969c-fd1ce77256d8 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.166533 May 19 12:44:54.063912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1702/3285] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:53 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:54.249055 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-b98884e0-19f7-440f-b21f-7191e01ef8ec req-4a57d922-ef0c-471f-85d0-5013a833c23b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01069733-00c3-4faa-a755-f7e224ade18a", "name": "network-vif-plugged", "status": "completed", "tag": "0ed79937-38b1-45ad-8dee-edbb5ccb7ea4"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:54.257263 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-b98884e0-19f7-440f-b21f-7191e01ef8ec req-4a57d922-ef0c-471f-85d0-5013a833c23b service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:54.257756 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-b98884e0-19f7-440f-b21f-7191e01ef8ec req-4a57d922-ef0c-471f-85d0-5013a833c23b service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:54.268405 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-b98884e0-19f7-440f-b21f-7191e01ef8ec req-4a57d922-ef0c-471f-85d0-5013a833c23b service nova] Creating event network-vif-plugged:0ed79937-38b1-45ad-8dee-edbb5ccb7ea4 for instance 01069733-00c3-4faa-a755-f7e224ade18a on hv-cinder-80403 May 19 12:44:54.273415 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-b98884e0-19f7-440f-b21f-7191e01ef8ec req-4a57d922-ef0c-471f-85d0-5013a833c23b service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.146268 May 19 12:44:54.273988 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1584/3286] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:44:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:55.084427 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:55.085281 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:55.092781 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:55.093248 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:55.094037 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:55.094479 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:55.228785 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:55.239094 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-aee2601a-f12e-4963-a9cb-6f3b0b6f7274 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1367 microversion: 2.1 time: 0.160815 May 19 12:44:55.239667 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1703/3287] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:55 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:56.255585 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell0:SELECT=6 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:56.262926 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:56.263746 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:56.271820 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:56.272340 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:56.273257 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:56.273759 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:56.408189 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:44:56.418154 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-6296e699-a4e5-4e81-add5-3c138458aedb tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1557 microversion: 2.1 time: 0.164265 May 19 12:44:56.418717 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1585/3288] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:44:56 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1557 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:44:56.434444 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:44:56.434920 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:44:56.441751 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:56.442184 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:56.442930 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:56.443357 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:56.519944 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-594acbbd-9199-4faa-87ca-095aa12cc9f6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089297 May 19 12:44:56.520664 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1704/3289] 10.222.0.35 () {58 vars in 1274 bytes} [Thu May 19 12:44:56 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:44:57.484893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-dab39318-d17a-4f1d-9c8e-4db7874ef8c7 req-4431d126-8b65-4083-995c-981da502f127 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01069733-00c3-4faa-a755-f7e224ade18a", "tag": "0ed79937-38b1-45ad-8dee-edbb5ccb7ea4"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:44:57.494218 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-dab39318-d17a-4f1d-9c8e-4db7874ef8c7 req-4431d126-8b65-4083-995c-981da502f127 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:44:57.494730 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-dab39318-d17a-4f1d-9c8e-4db7874ef8c7 req-4431d126-8b65-4083-995c-981da502f127 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:44:57.508277 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-dab39318-d17a-4f1d-9c8e-4db7874ef8c7 req-4431d126-8b65-4083-995c-981da502f127 service nova] Creating event network-changed:0ed79937-38b1-45ad-8dee-edbb5ccb7ea4 for instance 01069733-00c3-4faa-a755-f7e224ade18a on hv-cinder-80403 May 19 12:44:57.513312 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-dab39318-d17a-4f1d-9c8e-4db7874ef8c7 req-4431d126-8b65-4083-995c-981da502f127 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032142 May 19 12:44:57.513912 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1586/3290] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:44:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:45:33.711683 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:33.712919 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG dbcounter [-] [117533] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:33.719028 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:33.719821 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:33.729971 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:33.730416 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:33.731159 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:33.731582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:33.757176 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-e575d574-95f1-45a2-ba4d-149090d41e34 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/ips" status: 200 len: 150 microversion: 2.1 time: 0.047719 May 19 12:45:33.757732 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1705/3291] 10.222.0.35 () {58 vars in 1247 bytes} [Thu May 19 12:45:33 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/ips => generated 150 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:45:35.097891 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ae143ca5-5fbe-4fa8-953e-c575198647a4"}} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:45:35.099642 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:35.103106 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=1 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:35.103951 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG dbcounter [-] [117532] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:35.114326 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:35.115047 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:35.116212 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:35.116898 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:35.391461 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-51cfc263-3ca7-4cbf-91e1-9f0420010336 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "POST /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.300683 May 19 12:45:35.392063 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1587/3292] 10.222.0.35 () {60 vars in 1322 bytes} [Thu May 19 12:45:35 2022] POST /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments => generated 194 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:45:43.055771 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:43.056353 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:43.062419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:43.062854 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:43.063591 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:43.064013 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:43.164482 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.objects.instance [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lazy-loading 'services' on Instance uuid 01069733-00c3-4faa-a755-f7e224ade18a {{(pid=117533) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 19 12:45:43.243640 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-5dd3cebe-eaf7-4787-a21e-12506177a15f tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments/ae143ca5-5fbe-4fa8-953e-c575198647a4" status: 202 len: 0 microversion: 2.1 time: 0.192611 May 19 12:45:43.244288 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1706/3293] 10.222.0.35 () {58 vars in 1415 bytes} [Thu May 19 12:45:43 2022] DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments/ae143ca5-5fbe-4fa8-953e-c575198647a4 => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:45:43.254020 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:43.255092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:43.262074 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:43.262519 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:43.263262 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:43.263751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:43.298101 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-1cd5d4dd-073b-44a6-9da1-21c75f0ede51 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049036 May 19 12:45:43.298668 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1588/3294] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:45:43 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:45:44.315210 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:44.316893 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:44.324606 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.325068 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.325815 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.326238 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.355083 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-7db5ccd9-445d-448b-afe4-736a7d837b04 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048151 May 19 12:45:44.355671 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1707/3295] 10.222.0.35 () {58 vars in 1301 bytes} [Thu May 19 12:45:44 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 19 12:45:44.417763 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:44.418374 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:44.426709 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.427205 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.428344 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.429145 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.464301 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Going to try to terminate instance {{(pid=117532) delete /opt/stack/nova/nova/compute/api.py:2664}} May 19 12:45:44.547547 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9392ca05-8dc4-400e-ae6c-41eddb9d9fff tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 204 len: 0 microversion: 2.1 time: 0.135284 May 19 12:45:44.548216 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1589/3296] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:45:44 2022] DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 19 12:45:44.557448 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:44.558044 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:44.564137 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.564582 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.565419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:44.565800 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:44.704090 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:45:44.712743 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-4ba1d336-7851-4706-b4ac-e6b422eeb1f0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1682 microversion: 2.1 time: 0.159703 May 19 12:45:44.713280 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1708/3297] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:44 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1682 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:45:45.729925 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:45.730651 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:45.738083 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:45.738526 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:45.739278 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:45.739703 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:45.880801 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.policy [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117532) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:45:45.888871 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-3a908660-09b9-4a0f-84e2-9c008744f675 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1682 microversion: 2.1 time: 0.163272 May 19 12:45:45.889447 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1590/3298] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:45 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1682 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:45:46.906461 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:46.907020 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:46.914109 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:46.914552 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:46.915318 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:46.915763 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:46.977833 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.policy [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aad8b1af5994d7b9de3a8fb0be556cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0a91a3d6a14259b2d03881fdab4f9f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117533) authorize /opt/stack/nova/nova/policy.py:199}} May 19 12:45:46.985308 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-3d4e742d-a1db-47aa-a48a-3446daa721ac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 200 len: 1615 microversion: 2.1 time: 0.082816 May 19 12:45:46.985871 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1709/3299] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:46 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 1615 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 19 12:45:47.075751 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [req-5e3469fa-e0a0-4fa8-b979-10bc1dd88a11 req-a909601a-8a8b-4553-a5ce-5e309f316212 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01069733-00c3-4faa-a755-f7e224ade18a", "name": "network-vif-deleted", "tag": "0ed79937-38b1-45ad-8dee-edbb5ccb7ea4"}]} {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 19 12:45:47.084888 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-5e3469fa-e0a0-4fa8-b979-10bc1dd88a11 req-a909601a-8a8b-4553-a5ce-5e309f316212 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:47.085296 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [req-5e3469fa-e0a0-4fa8-b979-10bc1dd88a11 req-a909601a-8a8b-4553-a5ce-5e309f316212 service nova] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:47.096466 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.compute.server_external_events [req-5e3469fa-e0a0-4fa8-b979-10bc1dd88a11 req-a909601a-8a8b-4553-a5ce-5e309f316212 service nova] Creating event network-vif-deleted:0ed79937-38b1-45ad-8dee-edbb5ccb7ea4 for instance 01069733-00c3-4faa-a755-f7e224ade18a on hv-cinder-80403 May 19 12:45:47.101268 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [req-5e3469fa-e0a0-4fa8-b979-10bc1dd88a11 req-a909601a-8a8b-4553-a5ce-5e309f316212 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029129 May 19 12:45:47.102092 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1591/3300] 10.222.0.35 () {64 vars in 1317 bytes} [Thu May 19 12:45:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 19 12:45:48.002451 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:48.003038 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:48.010429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:48.010879 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:48.011716 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:48.012075 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:48.028302 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a HTTP/1.1 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:45:48.030016 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:48.031087 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '01069733-00c3-4faa-a755-f7e224ade18a'} May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:48.031698 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:48.032670 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:48.033528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:48.033528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-432f0c30-5a03-43ff-9179-d79af0d192f7 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 404 len: 111 microversion: 2.1 time: 0.032887 May 19 12:45:48.033528 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1710/3301] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:47 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:48.041384 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-abda620b-e486-4c14-b15a-26a4a7feaaac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '.version_select at 0x7fe1fff98430>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:48.062788 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-abda620b-e486-4c14-b15a-26a4a7feaaac tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1811807841" status: 202 len: 0 microversion: 2.1 time: 0.025722 May 19 12:45:48.063386 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1592/3302] 10.222.0.35 () {58 vars in 1220 bytes} [Thu May 19 12:45:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1811807841 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 19 12:45:50.109092 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:50.109571 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:50.117030 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:50.117474 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:50.118306 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:50.118734 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:50.141209 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a HTTP/1.1 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:45:50.143585 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.144864 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '01069733-00c3-4faa-a755-f7e224ade18a'} May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:50.145529 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.146621 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:50.147554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:50.147554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-04d0b239-f199-42df-8aac-4a9f9958b7b3 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 404 len: 111 microversion: 2.1 time: 0.039766 May 19 12:45:50.147554 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1711/3303] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:45:50 2022] DELETE /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:50.154508 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:50.155259 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 01069733-00c3-4faa-a755-f7e224ade18a] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:50.162170 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:50.162608 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:50.163552 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:50.164011 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:50.177987 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a HTTP/1.1 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:45:50.179583 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.181065 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '01069733-00c3-4faa-a755-f7e224ade18a'} May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:50.181733 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. May 19 12:45:50.183012 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:50.184258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 01069733-00c3-4faa-a755-f7e224ade18a could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:50.184258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-7216b06a-e69f-4cba-8ab0-2cdffb65cb93 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a" status: 404 len: 111 microversion: 2.1 time: 0.030875 May 19 12:45:50.184591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1593/3304] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:50 2022] GET /compute/v2.1/servers/01069733-00c3-4faa-a755-f7e224ade18a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:51.337429 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:51.337906 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:51.344765 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.345211 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.345962 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.346386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.360085 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b HTTP/1.1 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:45:51.361645 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.363212 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '7b74f125-b188-4751-baed-8d0bb14ded3b'} May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:51.363912 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.365231 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.366419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:51.366419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-9ca822a7-d3ae-477d-8dd3-9110feeaa1a6 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 404 len: 111 microversion: 2.1 time: 0.029826 May 19 12:45:51.366419 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1712/3305] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:45:51 2022] DELETE /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:51.374557 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:51.375172 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 7b74f125-b188-4751-baed-8d0bb14ded3b] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:51.384593 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.385071 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.385813 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.386258 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.399686 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b HTTP/1.1 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:45:51.401228 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.402395 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '7b74f125-b188-4751-baed-8d0bb14ded3b'} May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:51.403072 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. May 19 12:45:51.404211 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.405317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 7b74f125-b188-4751-baed-8d0bb14ded3b could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:51.405317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-9640ae8e-349a-4cf2-9370-4e835bc90cf4 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b" status: 404 len: 111 microversion: 2.1 time: 0.034338 May 19 12:45:51.405317 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1594/3306] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:51 2022] GET /compute/v2.1/servers/7b74f125-b188-4751-baed-8d0bb14ded3b => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:51.411059 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:51.411548 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:51.417881 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.418358 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.419102 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.419531 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.433173 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 HTTP/1.1 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:45:51.434706 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.436039 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': 'aad342e1-3718-42b8-8416-e8e0c57db755'} May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:51.436954 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.438111 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:51.439276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:51.439276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-a895f902-6c33-456a-9789-abcfbc45420a tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 404 len: 111 microversion: 2.1 time: 0.028888 May 19 12:45:51.439276 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1713/3307] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:45:51 2022] DELETE /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:51.446564 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:51.447167 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: aad342e1-3718-42b8-8416-e8e0c57db755] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:51.453544 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.454017 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.454789 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:51.455210 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:51.470986 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 HTTP/1.1 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:45:51.472471 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.473660 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': 'aad342e1-3718-42b8-8416-e8e0c57db755'} May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:51.474338 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. May 19 12:45:51.475591 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:51.476582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance aad342e1-3718-42b8-8416-e8e0c57db755 could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:51.476582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-a79f3260-94db-4f64-9ad2-d062461aefa0 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755" status: 404 len: 111 microversion: 2.1 time: 0.033117 May 19 12:45:51.476582 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1595/3308] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:51 2022] GET /compute/v2.1/servers/aad342e1-3718-42b8-8416-e8e0c57db755 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:52.617608 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:52.618228 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.compute.api [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117533) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:52.625086 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:52.625534 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:52.626275 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:52.626696 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG oslo_concurrency.lockutils [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:52.640386 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.wsgi [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: request: DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa HTTP/1.1 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept: application/json May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Accept-Encoding: identity May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Connection: close May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Length: 0 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Content-Type: application/json May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Host: 10.222.0.35 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Openstack-System-Scope: None May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: User-Agent: python-urllib3/1.26.9 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Id: None May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Domain-Name: None May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Identity-Status: Confirmed May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Is-Admin-Project: True May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Id: default May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Domain-Name: Default May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Role: member,reader May 19 12:45:52.641968 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Roles: member,reader May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Id: default May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Domain-Name: Default May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: method: > May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: exception: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.643151 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: action_args: {'id': '4440209e-114a-4e00-866c-006bba2d62fa'} May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: {{(pid=117533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:52.643848 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.644803 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: ERROR nova.api.openstack.wsgi May 19 12:45:52.645748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: DEBUG nova.api.openstack.wsgi [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. {{(pid=117533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:52.645748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO nova.api.openstack.requestlog [None req-cf1d7f10-392e-4c5a-9114-7672e9427654 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 404 len: 111 microversion: 2.1 time: 0.030812 May 19 12:45:52.645748 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: [pid: 117533|app: 0|req: 1714/3309] 10.222.0.35 () {58 vars in 1238 bytes} [Thu May 19 12:45:52 2022] DELETE /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:45:52.651840 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Calling method '>' {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 19 12:45:52.652376 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.compute.api [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] [instance: 4440209e-114a-4e00-866c-006bba2d62fa] Fetching instance by UUID {{(pid=117532) get /opt/stack/nova/nova/compute/api.py:2920}} May 19 12:45:52.659247 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:52.659792 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:52.660549 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 19 12:45:52.661016 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG oslo_concurrency.lockutils [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Lock "fdac0748-340c-456f-b8b7-e62d6fcd543f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117532) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 19 12:45:52.674625 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.wsgi [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] HTTP exception thrown: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Request method failure captured: May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: request: GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa HTTP/1.1 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept: application/json May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Accept-Encoding: identity May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Connection: close May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Length: 0 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Content-Type: application/json May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Host: 10.222.0.35 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Openstack-System-Scope: None May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: User-Agent: python-urllib3/1.26.9 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Auth-Token: gAAAAABihjraa1eQDcPlebmeXxiQt9mCVb-hHdJy0C2IYgfdqMJZ8oLrsugmciWwhgGYWuRLMoYATE_IlBrQ0JEMLef-nqeGebtcBZOEKjhMPe2IHmY85IoGjNrNxZhqhomFfnyjGb3wO6vi96A86sDbrsj_e8nmQt0TF_sUP0EprFoRj_hzOH0 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Id: None May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Domain-Name: None May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Identity-Status: Confirmed May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Is-Admin-Project: True May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Id: default May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Domain-Name: Default May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Project-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Role: member,reader May 19 12:45:52.676115 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Roles: member,reader May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0a91a3d6a14259b2d03881fdab4f9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0a91a3d6a14259b2d03881fdab4f9f"}]}] May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Id: 0f0a91a3d6a14259b2d03881fdab4f9f May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-45255492 May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Id: default May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Domain-Name: Default May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Id: 1aad8b1af5994d7b9de3a8fb0be556cf May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: X-User-Name: tempest-AttachVolumeNegativeTest-45255492-project May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: method: > May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: exception: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.677311 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: action_args: {'id': '4440209e-114a-4e00-866c-006bba2d62fa'} May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: {{(pid=117532) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 19 12:45:52.677893 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. May 19 12:45:52.679119 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: ERROR nova.api.openstack.wsgi May 19 12:45:52.679944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: DEBUG nova.api.openstack.wsgi [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] Returning 404 to user: Instance 4440209e-114a-4e00-866c-006bba2d62fa could not be found. {{(pid=117532) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 19 12:45:52.679944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO nova.api.openstack.requestlog [None req-ae7ad6ea-d1d9-4ed8-a485-17c5338d1440 tempest-AttachVolumeNegativeTest-45255492 tempest-AttachVolumeNegativeTest-45255492-project] 10.222.0.35 "GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa" status: 404 len: 111 microversion: 2.1 time: 0.029614 May 19 12:45:52.679944 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: [pid: 117532|app: 0|req: 1596/3310] 10.222.0.35 () {58 vars in 1235 bytes} [Thu May 19 12:45:52 2022] GET /compute/v2.1/servers/4440209e-114a-4e00-866c-006bba2d62fa => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 19 12:48:53.646176 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:48:53.647150 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:48:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:48:54.150022 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:48:54.150309 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:48:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:50:15.391783 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:50:15.392738 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:50:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:50:25.879348 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:50:25.880515 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:50:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:50:46.825221 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:50:46.826249 dv-cinder-804035-46-iscsi devstack@n-api.service[117533]: Thu May 19 12:50:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 19 12:50:51.414073 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 19 12:50:51.414725 dv-cinder-804035-46-iscsi devstack@n-api.service[117532]: Thu May 19 12:50:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!